Confirm Action

Are you sure you wish to do this?

Confirm Cancel
Member Login
Site Notices
1/25/2018 7:38:29 AM
Posted: 9/14/2010 7:29:27 PM EST
Here's the deal:

I need to build several hundred Cognos 7.x/8.x .mdc cubes overnight several times a month, and getting them done ASAP is a huge priority.
If your not familiar with OLAP cubes, just think lots and lots of read/writes. I mean lots of them. No, not that many, several times more than that.

Currently, a Dell powervault RAID array is the "swap disk" for processing (10 15K RPM disks in RAID 10, Ultra SCSI 320) (Yes, I got a $12,000 swap disk budgeted at the time).

Currently, the whole process is taking 12-14 hours, and disk I/O is the bottleneck. And 12-14 hours is way too fucking long.

What I need is the ultimate in HDD I/O. Lag and latency are my sworn enemy. I seriously don't care about recovery - once the processing is done, the results will be copied elsewhere. If I lose power in the middle of the process, I have bigger concerns - like WTF I just lost power.

OCZ's Z drives were looked at and rejected from several reviews of lackluster smaller size read/write performance.

FusionIO and HyperDrive5's are at the top of my list right now. Are there other options out there that handle random R/W faster, or almost as fast for less $?

Link Posted: 9/14/2010 7:43:02 PM EST
You're not going to beat a Hyperdrive since it is RAM based. There may be alternatives with the same technology and lower cost.

Perhaps your process is a bottleneck too?
Link Posted: 9/14/2010 7:46:49 PM EST
[Last Edit: 9/14/2010 7:49:59 PM EST by djkel]
Flash drives on a SAN are currently the bomb for disk I/O

Cost... well, if you have to ask...

Edit: have you tried 10k SAS drives in a RAID 10? Ultra320 SCSI is getting rather elderly these days.
Link Posted: 9/14/2010 7:53:49 PM EST
How much space do you need? You need to switch to solid state.
Link Posted: 9/14/2010 7:56:14 PM EST
[Last Edit: 9/14/2010 7:56:22 PM EST by adi]
Depends on budget and size requirements.

high cost: raid and SSD, although there are some long term issues with running SSD in raid.
lower cost: 15k 6g SAS array
fun low cost: ZFS array - people have built ZFS arrays with 7200rpm drives that bump 3200MB/s writes

I would think your bottleneck right now would be the 320 SCSI, since all drives share bandwidth per channel. If you are stuck to certain OS, you could build a SAN/iSCSI server and bond some GB links if local storage is too complex to build.
Top Top