Posted: 6/15/2010 6:26:59 PM EDT
|
For the past couple years I have had my own business on evenings/weekends doing photography. My PC has served me well for that, but it is time for an upgrade. RAW image files are getting larger and I am starting to venture down the expensive path of video work. Right now I have a Intel Q6600 with 2GB of RAM and some individual hard drives. The rest of my system is good enough for what I do. I really want to build a new system - i7, 12GB RAM, etc - but the total cost for the build I want is $3300+/- and there are others things I would rather get right now... So my two serious points of performance impact would be my disks and RAM. Obviously upgrading to 8GB of RAM on my system is going to make a huge change... and installing newer/faster hard drives with a good quality RAID controller (Areca) I can get more throughput and IOPS. Just updating my drives and RAM saves me about 50% of the cost of my system... Is my Q6600 going to be a major performance setback? Should I just bite the bullet and invest in a 100% new build? Is there another Socket 775 CPU that I should install to act as a stop-gap between my Q6600 and the i7? |
|
No processors out there that are going to make huge differences without going to the i5/i7 line.. You could easily get another 400Mhz out of the 6600, but I don't know if I would go down that road on a work machine. I have a QX6700 about 900MHz higher than stock on good air cooling running 100% stable. You could bump it up to one of the higher end Yorkfields but I personally don't think the performance increase is worth the $250+.
8GB of 1066MHz is going to be a HUGE bump in performance for you, and adding a SSD drive for the OS/programs and 3-4 short stroked (or not) drives in RAID 5 will scream. I mention short stroking the drives because if you only need say ~1TB internal, 4 1TB short stroked down to 3-400GB in RAID 5 would be extremely fast and reliable, just a bit more expensive per GB. What motherboard does it have right now? There are some out there with pretty good RAID controllers. Depending on if you actually need a separate RAID controller, less than a grand will really boost the machine for what you're doing with it. |
|
Quoted: What motherboard does it have right now? There are some out there with pretty good RAID controllers. Depending on if you actually need a separate RAID controller, less than a grand will really boost the machine for what you're doing with it. GIGABYTE GA-EP45-DS3L - no RAID... and come to think of it - not enough PCI-E slots for a RAID card either... might as well build the whole system! |
|
I have built three for my home office and several for clients. I spend most of my time working with lightroom, premiere pro, after effects, encore and photoshop.
What I can tell you is that you shouldn't need to spend $3300 to build a decent system for video editing. $3300 should almost get you a six-core extreme edition I7 system going. Here's some of my advice for what you do and don't need: Hard drives: get several BIG hard drives, but don't feel like you need insane RAID setups. ALL of my video editing is done via single hot-swap drive bays. No RAID, no speed issues. I have been using 1.5TB Seagate and 2TB Hitachi drives. Everything gets backed up on a regular basis, but not having mirroring can be a bit of a concern. Back up to RAID 5 server, plus external drive. RAM: 12GB is pushing it. Depending on what you're doing, you can easily be pushing 10+ on a regular basis. I would highly recommend not getting less than 12. For example, right now I'm 4 hours into a DVD transcode: 
Processor: If you do much rendering or transcoding, more faster cores will cut the time down tremendously. If you are doing simple and/or short projects, they will take less time and thus you might not care. I typically estimate 4x realtime to render out video feeds using a 3.2ghz I7. On a 5 minute video this isn't a big deal... on a 2 hour video, however... IMO, you can get by skimping on processor and MAYBE hard drives, if you don't need to keep lots of footage for a long period of time (tape sucks, but it is small and easy way to backup video). Look into a good I7 motherboard with plenty of SATA connectors (and an Esata if you can for fast backup to external), 12GB of RAM, and then go bare necessities on everything else (besides cooling). Estimating based off my last video build about 3 weeks ago: Good mobo $275 (2) 2TB HD (in raid 1) $300 I7 2.8Ghz $290 Basic video card $125 DVD RW $30 PSU $75 RAM $400 Case (Raidmax Smilodon... one of my favorites) $80 Win7 Ultimate OEM $175 Use your old mouse, keyboard and monitor. Total approx: $1750ish This will handle video pretty well. You may need to tack on more HDs in the future (always backup, RAID is NOT a backup!). |
|
If you're backing up to RAID 5, why not just use it for primary storage and an external for backup? There's no point to having a backup storage be faster than your primary storage.
Personally, if I'm spending money on a decent built I'm not going to skimp on storage because 'it's not needed'. Going from straight drives to SSD or RAID 0/5/6 is noticeable. |
|
Quoted:
If you're backing up to RAID 5, why not just use it for primary storage and an external for backup? There's no point to having a backup storage be faster than your primary storage. Personally, if I'm spending money on a decent built I'm not going to skimp on storage because 'it's not needed'. Going from straight drives to SSD or RAID 0/5/6 is noticeable. RAID5 is typically pretty slow. Check your benchmarks. It is good for redundancy and not losing a lot of space for parity... but not for speed. If you want speed, go RAID10 (mirror + stripe). Any RAID build also presents potential issues if you have a computer hang, lose power or need to replace a drive. The whole RAID array has to check/rebuild, which further slows you down. Then you get into the quality of RAID you actually get with cheap/integrated controllers, needing battery backups to avoid rebuilds, and a good little collection of other potential headaches... That's why I use it for backup and not production use (my "data" drive is R10, but I still edit off of single hot-swappable disks). I do, however have a SSD for my OS and programs, which makes that portion of things zippy... but in all honesty, on a budget, they're still too pricey for what I was recommending to him. If you have the extra funds, they're nice, tho. I would also add... only having one backup is not a good idea for anything you can't easily replace, if you ask me. So having a backup server PLUS an external drive(s) (which you would ideally move off location) would be the bare minimum for backups IMO. |
|
Quoted: If you're backing up to RAID 5, why not just use it for primary storage and an external for backup? There's no point to having a backup storage be faster than your primary storage. Personally, if I'm spending money on a decent built I'm not going to skimp on storage because 'it's not needed'. Going from straight drives to SSD or RAID 0/5/6 is noticeable. My drive layout was going to be Two Raptors in R1 for OS/Apps Two Raptors in R0 for a working drive - all active projects copied on their for speed Maybe One SSD for an application scratch disk... apps like Photoshop, etc love fast speeds on their scratch disk and who cares if the drive fails (no RAID) - it is all temp data. Five 1TB drives in R5 for less active data, but data I still want some good performance out of (better than Firewire externals) (While R5 is not the fastest RAID out there, 5 hard drives in R5 is faster than 5 standalone drives...) I really want SSD - I've got a few at work and they are really impressive. But they are so new right now and the cost premium is so high... ehh... plus RAID support of TRIM, etc is coming soon to Areca and 3Ware controllers. Speaking of RAID controllers –– the Areca I want to buy has a backup battery and 2GB on-board cache - So a lot of issues "cheap" controllers have go away. For backup I have a few different mirrored externals that I cycle out, a NAS on my LAN, and then some files I upload to an online backup place. |
|
Yeah RAID 5 has a slower random write then RAID 0, but fairly comparable continuous write and read, which is why you use it for data and not program/OS storage. RAID 5 is still significantly faster than RAID 1. Yeah 10 would be ideal, but it's also expensive. I advocate 5 because it's faster than 1 and safer than 0 while still being cheap...
I run RAID on all of my personal computers and every storage system I've worked with in the industry. I have never had to rebuild a RAID for any reason beyond losing a drive. And yeah, rebuilding a RAID is going to slow him down, but how often will he have a drive fail? Once every 5-10 years? Maybe shorter if he's unlucky. Having the original data in RAID, and an off-site backup should be sufficient, even if the off-site isn't RAID. If you have ~50% of your disks fail at one time you have major major issues. I mean if RAID 1/0 or 5/6 on-site and 5/6 off-site is good enough for the government I don't see why it isn't overkill for an on the side business. Data lose due to power outages aren't a problem because of UPSs. Not using one on a work machine? Then you deserve to lose your data. Anecdotally, I run RAID 0 in all three of my good machines and I abuse the crap out of them and I have never lost any data. Hard shut-downs, improper start ups. I've moved an array from one computer to another and from one controller to another. They've always been recognized immediately and have never required a rebuild. The only drive that I've ever personally lost that was less than 10 years old was a 500GB WD that I accidentally submerged in liquid coolant.  Other then that, I've only seem a handful of drives fail at work out of the few thousand that a running 24/7 in the last 4 years..
I'm not trying to be argumentative btw, just my experience is that they aren't as fragile as they're made out to be... |
|
Quoted:
Quoted:
If you're backing up to RAID 5, why not just use it for primary storage and an external for backup? There's no point to having a backup storage be faster than your primary storage. Personally, if I'm spending money on a decent built I'm not going to skimp on storage because 'it's not needed'. Going from straight drives to SSD or RAID 0/5/6 is noticeable. My drive layout was going to be Two Raptors in R1 for OS/Apps Two Raptors in R0 for a working drive - all active projects copied on their for speed Maybe One SSD for an application scratch disk... apps like Photoshop, etc love fast speeds on their scratch disk and who cares if the drive fails (no RAID) - it is all temp data. Five 1TB drives in R5 for less active data, but data I still want some good performance out of (better than Firewire externals) (While R5 is not the fastest RAID out there, 5 hard drives in R5 is faster than 5 standalone drives...) I really want SSD - I've got a few at work and they are really impressive. But they are so new right now and the cost premium is so high... ehh... plus RAID support of TRIM, etc is coming soon to Areca and 3Ware controllers. Speaking of RAID controllers –– the Areca I want to buy has a backup battery and 2GB on-board cache - So a lot of issues "cheap" controllers have go away. For backup I have a few different mirrored externals that I cycle out, a NAS on my LAN, and then some files I upload to an online backup place. Want to fund my next build Seems like you have a good plan ahead of you, and it'll be like Christmas in June when 10+ drives show up on the doorstep. I know it was for me when I had 4 Raptors show.
|
|
Either you're extremely lucky, or I'm extremely unlucky.
Any time you improperly shut down or lose power, the array will check and (if needed) rebuild itself. At least on every different controller (plus all windows software raid) setups that I have ever worked with. You do not have to manually do anything for this to happen... but it is happening, and the array is even slower when this is going on. 4+ multi-terabyte drives being checked/repaired takes a long time. Also, my experience with R5 has not led to ANY good write speeds (literally it is abysmal) and no significant read speed benefit. I mean, really... editing 1080p video you will rarely be pushing much more than 50Mbit /sec (not MByte). My solo hard drives transfer over 100MByte /second (Usually peak around 120). If R5 would actually be an improvement... what benefit would you get over 100+ MB/sec transfer for editing a sub-100Mbit video feed? I just don't see any benefit, but there is a huge risk. For example, your card dies... you have to have a replacement card on hand to even see your data again. R10 is 50% overhead, where R5 is 1 drive overhead (4 drives = 25% overhead), I'm not sure how that would be considered expensive over other forms of RAID... The OP (summarized) stated "It's expensive to build a new system (at $3300), maybe I should spend half and upgrade my C2Quad." My recommendation should have quite easily satisfied the needs of HEAVY video editing and still be about half the cost of what he suggested. Since it now sounds like we're on more of a fun/geeky tech build, rather than an effective & cost efficient solution, my suggestion may be wasted. And for the record, there's nothing wrong with fun/geeky builds... I do it too. 
|
|
Maybe I am just lucky. On my best machine I've moved my RAID 0 of Raptors from an Intel ICH7R to ICH9R to an Adaptec controller back to the ICH9R and every time it's just go into the settings set up a RAID 0, and voila my array is back to the way it was. That machine is overclocked 35%+ and has hard shutdowns all the time. Every other RAID that I'm around are production machine so they don't go down unexpectedly.
Surprised your experience with R5 has maxed out around 50MB/s. Should be up around 170 average... I just meant more expensive in that 25% is the max R5 will be where R1 will always be 50% I've never used a software RAID... 
|
|
Quoted:
Maybe I am just lucky. On my best machine I've moved my RAID 0 of Raptors from an Intel ICH7R to ICH9R to an Adaptec controller back to the ICH9R and every time it's just go into the settings set up a RAID 0, and voila my array is back to the way it was. That machine is overclocked 35%+ and has hard shutdowns all the time. Every other RAID that I'm around are production machine so they don't go down unexpectedly. I use the Intel ICH (built in mobo) for the 3 systems at my office... perhaps you don't have the monitoring software installed? Cause they check (and slow down) if you sneeze wrong. Okay, not quite that bad... but if shutdown hangs and you have to manually power off, after you boot back up it is checking the entire contents of the array, unless you manually go in and tell it to stop. I actually just spent a couple of days fixing a glitch from one drive failing (on the R10 array). When the new drive was put in to replace the failed drive, the rebuild seemed to go fine, but every reboot files & indexes were corrupt and missing. Finally had to completely delete the array and recreate a new one to get it working right again. Surprised your experience with R5 has maxed out around 50MB/s. Should be up around 170 average...
My experience was actually a lot worse than 50MB /sec with 4 1TB drives in a R5 array. IIRC it was the ICH7 or 8... haven't bothered trying since then, as nothing I was doing would give decent writes, and that seemed to be pretty typical when reading other users experience at the time. Reads weren't nearly as horrible, but they weren't a significant improvement over a bare drive, either. If it should be pushing 170+ MB /sec I may have to try that out on the next machine I build. That would be a pleasant surprise. And less overhead would certainly be welcomed. But what I actually was trying to say (probably didn't come across properly) was that 1080p video will usually be near 50 Megabits a second for a good feed... unless you're working with uncompressed footage (or something similar), getting a disk performance boost for editing video footage won't make a difference, as a single bare drive will push over 100 megaBYTES (800 megabits) per second pretty easily. I just meant more expensive in that 25% is the max R5 will be where R1 will always be 50%
I've never used a software RAID... Gotcha. Although I think the ICH RAID is pretty much software (as opposed to a dedicated card which will be truly hardware based) RAID, which may be why you can swap controllers easily? I can tell you that with CERC and PERC cards as well as back in the adaptec 2940 days (yeah, that dates me ), it (unfortunately) wasn't that easy. And it wasn't quick or cheap to get it working again. 
The only reason I even bother running Raid nowadays is because I have two other similar systems sitting nearby that I know I can swap drive cages with to get the data off quickly if needed... otherwise I would probably be sticking with my previous experience and not bothering, except on backup systems. I think it may have a lot to do with age, and getting tired of messing with crap all the time. I used to enjoy spending hours upon end squeezing every little bit out of a computer just for kicks... now I find that I'd rather be making money or spending time relaxing instead. Somewhere along the line it stops being as fun as it used to be, and becomes work... work that you don't get paid for. |
|
If the controller doesn't suck R5 should be quicker than non-RAID. I mean a 4 drive R5 is basically a 3 drive R0 with a parity drive. You lose some throughput calculating the parity and what not, but still...
Quoted:
I think it may have a lot to do with age, and getting tired of messing with crap all the time. I used to enjoy spending hours upon end squeezing every little bit out of a computer just for kicks... now I find that I'd rather be making money or spending time relaxing instead. Somewhere along the line it stops being as fun as it used to be, and becomes work... work that you don't get paid for. I got to that point... So I moved to playing with dry ice cooling. Still couldn't get my QX6700 past 4GHz. Tried hard enough that I found out that Intel has an amazing RMA policy. 
On a side mote, my power just went out. 
|
|
Quoted:
If the controller doesn't suck R5 should be quicker than non-RAID. I mean a 4 drive R5 is basically a 3 drive R0 with a parity drive. You lose some throughput calculating the parity and what not, but still... Quoted:
I think it may have a lot to do with age, and getting tired of messing with crap all the time. I used to enjoy spending hours upon end squeezing every little bit out of a computer just for kicks... now I find that I'd rather be making money or spending time relaxing instead. Somewhere along the line it stops being as fun as it used to be, and becomes work... work that you don't get paid for. I got to that point... So I moved to playing with dry ice cooling. Still couldn't get my QX6700 past 4GHz. Tried hard enough that I found out that Intel has an amazing RMA policy.
On a side mote, my power just went out.
Hey... at least you're the lucky one. Not 5 minutes ago I was typing a response on the other sub-forum (photography) of arfcom that I frequent, and the *@#$)*ing Intel Matrix Storage Controller popped up that the array is degraded. Port 4 decided to go bye-bye. Same crap that happened a few weeks ago that caused constant corruptions (except it was port 5... even with 4 different disks on that port). If I have to re-format and rebuild again, I'm sending the motherboard back. |
| You are overthinking the RAID setup. If you must have RAID, throw two or three drives in RAID0 on a board with an Intel ICH10R southbridge. The performance is fine with RAID0. In reality, RAID does not have that much effect on the performance of a primary drive, as the seeking is mostly random and not the sequential that RAID0 excels at. RAID1 is entirely unnecessary unless you simply cannot afford to have any downtime in the event of a failure. You would be better served by a single fast SSD. |
Off topic: Me thinks you are using the various types of RAID in ways in wasn't meant to be used..  You are overthinking the RAID setup. If you must have RAID, throw two or three drives in RAID0 on a board with an Intel ICH10R southbridge. The performance is fine with RAID0. In reality, RAID does not have that much effect on the performance of a primary drive, as the seeking is mostly random and not the sequential that RAID0 excels at. RAID1 is entirely unnecessary unless you simply cannot afford to have any downtime in the event of a failure. You would be better served by a single fast SSD. Edit : I was kinda thinking the same thing. The quality/feature of a give raid array type are heavily influenced by the controller and management software for the controller. |