1 TibiByte Wet Dream - Part 2

Requisites and Contemplations

What do we need to realise the idea within a limited budget? Consider an average collection of a couple of dozens of DVD's, some hundreds of CD's, storage of all applications you use and the projects you make with them I think it's safe to say that a terabyte of storage should be enough for now, but growth must remain a possibility (of course) at all times.
A terabyte is 1024GiB or 1.048.576MiB, enough to (for example) store about 200 DVD's, or 17.500 high-quality MP3 CD's. The majority of the storage space will go to the DVD's anyway, because even a RAW picture is still about 10MiB, or a serious Reason-project between 50-100MiB.
So we're going to compose an array and server with 1TiB (starting) storage. A box which sole purpose is to store as much as possible as safe as possible. Load-baring won't matter much, because the number of simultaneous users will pulling or pushing data will probably never exceed 5.
Of course we're not going to run Windows on that machine, but Linux, BSD or (in my case) Solaris.
The main advantage (besides the obvious performance and price advantages) is that these Operating Systems are capable of creating and running a software RAID. The advantages of that against a hardware solution are numerous:

• Price: The Operating Systems are free, the hardware they require minimal. A hardware RAID-controller for just 4 harddisks will quickly lighten your wallet about € 500,-.
• Migratability: If you need to migrate the entire array to another platform or your controller dies on you, in the case of hardware RAID you'll have a serious problem getting your array back.
• Flexibility: There's no obligation to have identical hard disks. If you decide to take a bunch of 250GB harddisks now and you decide to add 500GB's later you'll lose half of the latter in the hardware RAID-case.
• Scaling: It doesn't matter how many harddisks or controllers are involved. In the hardware RAID-case you'll have to find controllers with the capability and add as much as you can or need à € 500,-.

All things considered I think for a single person or household the software RAID is the way to go.

The RAID Array
We want a minimum of 1TiB and that 1TiB has to be completely resistant to (mainly) the breaking down of a harddisk. It's obvious we're building a (software) RAID-array for this. This site has had a very good explanation of the various RAID "levels" for a long time now. At the time of writing the biggest available hard disk is 500GB, of which we will need 3 to get that one TiB with redundancy. In other words: we can't use a mirror. One could consider RAID 1+0 or 0+1, but that would be financially inefficient at best. So we've arrived more or less automatically at RAID 3, 5 or 6. After reading a good comparison between RAID 3 en 5 I've come to the conclusion that RAID 5 is the best choice in most cases, because our array has to be able to grow after all (the parity-disk will become a very serious bottleneck in RAID3), and because we're going to make it a file-server anyway (the extra overhead involved in the use of RAID5 is tolerable). RAID 6 is worth considering, but I personally don't believe 2 harddisks will quit at the exact same time without warning you well in advance.
Having made the choice of the RAID-level we can now compare the cost and number of harddisks. We will be taking SATA harddisks to allow for future growth, and if possible we'll take SATA-II extensions as well:

A few general remarks:

• Notice that manufacturers give their sizes in GigaByte, while I consequently use GibiByte. This conversion takes an extra harddisk in most cases. However the relationship and differences between the figures, especially the price per GiB, is only slightly affected by this.
• Prices are average estimates. Voor de 200, 300, 400 en 500GB the prices are based on Seagate Barracuda 7200.8's, see below. For the remainder the cheapest possible varieties for the given size. This is because with that amount of harddisks the harddisks themselves are not the bottleneck anyhow anymore, it's the entire I/O system. That's also why it won't particularly matter if we have SATA-II extensions or not.
  
• The only manufacturer manufacturing 320GB harddisks is Western Digital at this time.

The last couple of months or so Maxtors have been dying all over the place, so I'll be damned if I buy one of those again (which is a high price for Maxtor, which used to be my single brand of choice for years). Western Digital is banned because of previous experiences as well. I was going to put Hitachi's in. Ever since IBM sold it's harddisk unit to Hitachi it's been producing consistently good and innovative harddisks scoring high in the benchmarks and being known as reliable. A good friend of mine however pointed out that Seagate has a 5 year warranty on it's Barracuda 7200.8-series, while the others will only offer 3 years. Aside from that the Seagates are comparatively more silent than the Hitachi's in the available online benchmarks, and there's a good chance they produce less heat as well.
All things considered I've eventually chosen the Seagate Barracuda 7200.8 250GB's.
If you buy 5 at a time you seem to be able to get an extra discount, driving costs down further. In this case you'll have to shell out about € 700,- for the storage space alone, or € 600,- if you'll be happy with one harddisk less.

tags: storage  reliability  disk  terabyte  gigabyte  raid  

links: digg this    del.icio.us    technorati    reddit