Need Raid 5 Suggestions / Help

[SoonerToucan]

I plan on setting up a Sage Server that will support a nuber of clients and a large amout of storage. From what I have read on these forums Raid 5 Seems to be the best solution for TB sized configurations. Comparing the prices of drives 250GB($.60 / GB) vs. 300GB($.83 / GB) the 250 GB drives seem to be the better solution.

The downside to using the smaller drives seems to be:
More Power
More Heat
A bigger Raid card

To hit the TB level of storage using the 250GB drives I would need a RAID card that supports 8 devices and those seem pricy. This card seems to the card that has gotten the best reviews from you guys, but at $500 that is a large increase to the price of the project.

I was wondering if anyone out there had some good ideas on how to achieve the TB size array that I am looking for. I don't know a lot about RAIDS so forgive me. Would it be possible to get two cheaper raid cards that only support 4 drives and somehow use them together to form one large drive?

Anyway, thanks for the help and if you have any suggestions on hardware or setup I would appreciate them.

Thanks,
-ST

[stanger89]

Well, you've done your research, that's good. Let's look at your options:

300GB drives on a 4-port card:
1x 8506-4LP = 330
4x 300GB = 231x4 = 924
Total 900GB for $1254

250GB drives 8-port card
1x 9500s-8mi = 465 (Better to go with the 9500s since they will feature OCE so you can add drives later)
5x250GB = 5x150 = 780
Total 1TB for $1245

Basically the card is more, but you can save more than the difference in the cost of drives. The 9500s-8 has the added benefit that it will support Online Capacity Expansion (OCE)* so if at a later time you decide you need more space, you can just add a drive, and grow the array. Or alternatively you could build a smaller array now, and add drives as you need them.

*OCE isn't available yet, still listed as "Summer 2004"

[SoonerToucan]

Thanks for the help stranger. I will give a bit more insite into my thought process on this and ask a few more questions if you dont mind.

What my goal is to have a central repository for my PVR and my VOD needs. I have a large collection of DVDs I would like to archive in my server so I can bring them up on demand. My first thought was to use two or three 300gb drives not in an array and just add new ones as nessissary. From my research it seems that in the end the raid is a much better solution. I figure I could build the system this way, but I see it as a bit short-sighted. I hate to spend the money, but I think if I didnt do it this way I would be kicking myself down the road.

Assuming I go with the raid I have the following additional questions;

What processor requirements would I have for this setup? ( I was planning on a 2.8ghz P4)

How big of a power supply is needed? Also if you have any suggestions of powersupplies that support so many devices. I have been searching newegg, but I can't find any info on how many power connectors that the ps has.(planning on a 400w, eventually having 8 HDDs)

What RAM requirments would there be? (planning on 1gig)

Roughly how much space does one DVD take? (assuming I want to be able to access it through the sage interface)

With the addition of the raid card I assume I do not want a motherboard with onboard RAID support, correct?

You mentioned the OCE support and as I understand it the above mentioned card does support "on the fly" upgrading. I assumed this meant that if I wanted to add the final 3 drives at a later date I could do it easily without losing any stored information, corret?

Thanks,
-ST

[Grey_Goose]

The Highpoint RocketRaid 404 & 454 have gotten excellent reviews on Tom's Hardware & other independent reviewers. They're 8 drive cards for under $100.

[Rob]

Regarding space required for a DVD, I've got about 100 DVD's ripped and the average space required is 4.7 Gig. I've used DVDShrink to rip only the main movie (no extras or menus) and I've got little kids, so probably 60% of the 100 are 90 minute animated movies.

Regarding the OCE. Yes, once it is really supported by the 3ware card, you can extend the array without losing data.

[stanger89]

Originally posted by Grey_Goose
The Highpoint RocketRaid 404 & 454 have gotten excellent reviews on Tom's Hardware & other independent reviewers. They're 8 drive cards for under $100.

Those cards are crap IMO, they have no hardware support for RAID, they're basically IDE controllers with special drivers that do the RAID stuff in software.

Then there's the fact that you should NEVER build an array with drives on both the master and slave connections since if one drive dies it takes down the whole channel. RAID-5 can survive 1 drive failure but not two. (FWIW, the array MAY survive if a drive dies in a master/slave config).

[stanger89]

Originally posted by SoonerToucan
Assuming I go with the raid I have the following additional questions;

What processor requirements would I have for this setup? ( I was planning on a 2.8ghz P4)

For a media server, not much, 2.8 is way overkill. You shouldn't need anything more than about 1GHz.

How big of a power supply is needed? Also if you have any suggestions of powersupplies that support so many devices. I have been searching newegg, but I can't find any info on how many power connectors that the ps has.(planning on a 400w, eventually having 8 HDDs)

I've seen conflicting info on this, for example most people say go big (500W) but I've run 4 HDDs in my P4 2.4, Radeon 9500, PVR 250, with a 300W. So off to do some research...

I checked the WD site and these are the electrical specs for a WD2500JD:

Current Requirements
12 VDC
	Read/Write	450 mA
	Idle	470 mA
	Standby	23 mA
	Sleep	23 mA
5 VDC
	Read/Write	800 mA
	Idle	750 mA
	Standby	330 mA
	Sleep	200 mA
Power Dissipation
	Read/Write	12.80 Watts
	Idle	9.50 Watts
	Standby	1.90 Watts
	Sleep	1.30 Watts

So, by that, 8 HDDs would need about 64W average, but 102.4W Peak, so figure 150W for a little safety factor.

Now here's the important part, look at the current requirements, according to them you'll need a PS that will do:
10A on both the 5 and 12V rails (safety factor again).

With that in mind, and assuming an average PC load (my 2.4GHz HTPC draws about 200W) I'd say you'd be good with a high quality 400W power supply:
http://www.pcpowerandcooling.com/pro...cers/index.htm

What RAM requirments would there be? (planning on 1gig)

Again, for a server, 512 would be plenty.

Roughly how much space does one DVD take? (assuming I want to be able to access it through the sage interface)

Figure about 5GB for each movie, and if you have TV show type DVDs figure closer to 7 for those.

With the addition of the raid card I assume I do not want a motherboard with onboard RAID support, correct?

Yes, you don't need RAID on the motherboard, but it wouldn't hurt if it were there. Guess I'm saying pick a board you want and if you can get it without RAID, great, if not, it doesn't matter.

You mentioned the OCE support and as I understand it the above mentioned card does support "on the fly" upgrading. I assumed this meant that if I wanted to add the final 3 drives at a later date I could do it easily without losing any stored information, corret?

Yup, but you will need to use some software (like Acronis Disk Director) to expand the partition on the array.

Thanks,
-ST

[pdxcoop]

I just built a Raid 5 sage server. This is what I used:

5 250 gig Maxtor ATA 100 hard drives from Fry's. The go on sale every other week.

Then from ebay I bought the LSI Logic Mega RAID i4. This card could be had for 50-100.00.

Works great. No problems recording 2 shows at the same time and streaming 2 streams at the same time.

[zz5]

Have any of you ever looked into data recovery when using RAID?

I would like to use the RAID controller on my motherboard in RAID, but I haven't done so because of the increase risk of data loss (one drive goes down, you lose all 4 drives). I'm not sure of the name, but I think it might be JBOD mode, where data will be written on one disk when possible (as opposed to striping which writes to multiple disks for speed). This should make data recovery easier. I've read that's the reason people use this mode instead of striping. Is there software that could be used to recover from a failed disk? I looked into this once but didn't find out much.

[stanger89]

You're thinking of spanning. JBOD is essntially no-RAID

RAID-5 can handle a single drive failure, ie if one drive fails no data is lost. However you need to get a good card (IMO) if you plan on doing RAID-5. The array will even still work after a drive failure, but will be in degraded mode (slow and not fault tollerant) until you replace the failed drive and rebuild the array.

IMO RAID-0 definitely and Spanning (probably) have no place in a home computer or HTPC.

[SHS]

Perason I think RAID over kill after SageTV all ready dose support Multi-HardDrive Recording.
Don't for that thoses cards you boy are l look at need diff Mother Board in order to work 64 bit PCI slot.

[Opus4]

The advantage of RAID 5 is that you won't lose all the recordings on a drive if that drive happens to fail. It is pointless to run a backup program, since that would take up just about as much storage as the orignal recordings, so you waste an entire extra drive doing backup, while with RAID 5, you only lose 1 drive total to a form of backup data for all the other drives. You are still out of luck if 2 drives fail at the same time, but I don't think I've ever had that happen. One of these days I might consider RAID 5.

Oh -- and it may not just be TV recordings we are talking about. If you've ripped a bunch of DVDs or CDs, you don't need to worry as much about having to rip them again if a drive fails. I've now got all my ripped CDs stored on 3 separate drives because I don't feel like ripping them all yet again.

- Andy

[stanger89]

They work in 32-bit PCI slots too. Yeah it's a little overkill for Sage, but the added security of RAID-5 is nice when you're talking 1TB+ of data.

[SHS]

Yes but most important thing to keep mind that the 250, 350 in same system don't alway like RAID drive and has been know also have problem with SCSI setup to so I guest if you building this like Network Storage that should be fine.
But wouldn't it be eazyer to get and LaCie Bigger Disk or maybe an NAS (Network Attached Storage)

[stanger89]

I've had my 3ware 6400 in the same system as my PVR 250 for close to a year with 0 problems. Recorded to my WD2000JB most of the time though.

[zz5]

The reason I'm interested in RAID is to make disk space calculation easier. It's much easier for Sage to work with one volume. For example, let's say you want to record something that will take 10GB. If you have 4 hard drives, you have to make sure that Sage has 40GB free (because the worst case scenario would be 10GB free on each drive) unless you actually check the space remaining on each drive in Windows. When you try to figure out if you enough space a day or two ahead, it gets even more complicated, because Sage will use the partition with the largest free space, so you could have a recording that only needs 1GB scheduled before the 10GB recording, and the 40GB wouldn't be enough.

I don't really see many people complain about this problem. Does this bother anyone else? Does everyone else have so much free space that you just never have to think about this?

[Opus4]

One big drive would make that easier. I think a lot of people let SageTV automatically handle the space, allowing it to delete shows as needed to make room for new ones. In don't know how many people manage it themselves & let free space get fairly low. Because of some recent work I did on my system, I now have 500GB for recordings, but it has yet to fall below 200GB free because there just aren't that many shows I want to leave sitting around. If I know I'm not going to watch a show any time soon, I delete it.

- Andy

[SHS]

It depend Opus4 if your doing TV Episode then you need lot of space unless you get really lucky and don't get any bad clip, etc, etc.

[GlobusProject]

The best solution if you are going to store alot of movies over many years, is to go with the RAID 5 setup. I have 4 Maxtor 250 GB HDD on a 3Ware 8500 card running PIV 2.66 GHz and ATI Radeon 8500. I am not happy with the ATI Radeon, and am replacing it with a PVR 350 and adding Sage 2.0. But the storage situation is what is important in this discussion. I am building a couple of AMD 848/846 dual Opteron servers with the Tyan Thunder S2882. These servers will be used in a non dedicated cluster arrangement and when I am not using one of them for HTCC, I can use it to process video. I have found that the 3Ware cards are the way to go. With the 3Ware 8500 it can address 1 TB of storage and the 9500 cards can address 2 TB of data eachm (or more). I have a Vixel 335 SAN switch and plan to build a SAN with 2 GB/s fiber transcievers and HBA cards (LSI 7202P). This will put everything together, allowing for future growth. All of the arrays (because that is what you are buliding with 1 TB of data/file servers), can be connected and the data arrays can address these file servers in a fabric mesh type SAN structure. I have Mac G5, XServe, Suse LInux and Windows 2000 machines and have to allow them to access these file servers as they need over NFS or Samba. I would love to go to a pure Unix/Linux type solution but Windows is so convienient for Sage 2.0. In a few years we can do everything in Linux/Unix. The problems I have had are with various file systems that everyone must access. In using the 3Ware cards I have found that having a 1 TB or larger system is convienient to store video on the Windows box.

If you can afford it and you plan on storing alot of video on disk, then the 3Ware 9500s with OCE is the way to go. You would want to have RAID 5/0 if you can but RAID 5 is good if you can afford it (if two drives fail). You should think about backing this up to tape, if you want to protect yourself against complete failure. Although it is cheaper to back up to hard drives these days.

The Chenbro rack mount cases are the best on the market now and you can get them in 8, 12, 16 SATA HDD mobile rack and later put them together in a SAN later on. You can add more than ono 3Ware card to each Motherboard and run multple 12 port 3Ware 9500 controlers per 3-4 U rack spaces (with 16 SATA drives per 4 U.

This may seem like over kill but 5 years ago we were happy to have a 10 GB hard drive. Now we are talking 1TB. With the addition of the new HDD TV codecs (Built into Mac OS X presently) there will be 4 times more resoulution with the new format (data density???) than presently. That could mean 20 GB per movie of storage space required.

I can easily see a TIVO recording to your hard drives and everyone in the house able to access your hard drives from a TV to choose the movie that they want to see. Or your friend, over the internet, accessing your hard drive array and watching a movie at their house. (Comcast does it today, why not?).

If you are going to take the time to record and store this data, then you must plan on the large growth of the files you are going to store. But in any case, the 3Ware cards are the ones that you want and the way to go for the future.

It is also the cheapest way to go, with the most security for your data, and the one that will allow you to use this hardware with 64 bit hardware in the future.

I don't know the future, but with the 3Ware setup you can stream multiple streams over GB Ethernet (or 10 GB Ethernet) to multple TV's over your LAN. This is the future, and if you buy your equipment with this in mind, you will not have to upgrade every 18 months like we have done in the past. You can also use it as a file server over your LAN. There are plans to have 400 and 500 SATA HDD in the not too distant future, so you can sell the 4-5 HDD's and install the same amount of HDD to have a 2 TB array. It DEFINATELY makes sense to make a RAID server today and add to it as your storage needs increase. Using multilple 3Ware cards in a large case 4U with 16 mobile trays for 16 SATA drives is the way to go. Just add more 3Ware cards and SATA drives as your storage needs increase.

IBM is doing research on using electrons (flipping them up or down- 0 or 1) as bits. counters. In 10 years or less we could have such massive storage. But for now, think about a good 16 (or 8) SATA rack mount case and add to this with time, as your storage needs grow. This is the cheapest way to do this and if you put away a little money every month, you will have the storage that you will need to do this correctly. Putting all of your work onto 2 250 HDD and having one of them fail without RAID recovery is NOT something that you want to go through, trying to recover the hundreds of GB of data that you would lose.

Hope this helps.

[stanger89]

Originally posted by GlobusProject
I am building a couple of AMD 848/846 dual Opteron servers with the Tyan Thunder S2882.

Wouldn't you want to use 24x's in a 2-way setup?

With the 3Ware 8500 it can address 1 TB of storage and the 9500 cards can address 2 TB of data eachm (or more).

Actually it's 2TB per array on the 8000 series, and several Yotabytes on the 9000 series


Nice setup though, wow