hard drive controllers

[S_M_E]

Originally Posted by davephan

S M E,

What mobo can run 9 SATA 2 drives right of the mobo without adding a controller? Does the mobo have on board RAID?


Dave

Abit AB9-Pro (They make a quad core version of that too) and, yes, it supports RAID on 3 different on board controllers.

[stanger89]

Originally Posted by S_M_E

WHS isn't a redundant array, it's a drive extender that uses pooled drives. With RAID a 250G drive and a 1T Drive would give you nothing. As with my previous example, if you have 2 500G and a single 1T, you could duplicate the data but you couldn't with RAID.

The key there is "most" of the time and "should" be able to, sometimes you cant.

It's the "sometimes" people should be aware of.

Again semantics...

Technical distinction, and an important one if you are trying to understand how much data you can write to either option.

Larger arrays are going to need more than one drive handle parity.

You might want to use more drives, but you don't need to. You can create a RAID-5 array of 24 drives and only one drive's space will be used for parity information.

Only on duplicated shares. Duplication is on by default but it can be turned off on a per-share basis, RAID isn't as flexible.

Yes, but look at the OP. The OP wants a redundant storage solution: "I also need to build some sort of redundant array". Hence we're talking about redundant storage solutions.

If you just want a bunch of drives to show up as one location and don't care about redunancy, then there's no point going RAID in the first place. Hence, like I said I'm comparing Windows Home Server and RAID-5 in regards to their solution/implementation of redundancy (which is what the OP asked about). I'm not comparing RAID-5 to an operating system.

Not more than a mirror...

No, but that's the point, you go RAID-5 because the cost of mirroring is prohibitive.

Good catch, I was thinking 500G not 50G and as I said, you're right that with more data that advantage disappears.

We do intend to fill our storage right? If so, then WHS's folder duplication does not hold a capacity advantage over RAID-5, quite the opposite.

The advantage to WHS there is expandability. If you out grow that 6T and need more you can add more drives to WHS without having to buy 3 or 4 more drives and another controller.

That's what Online Capacity Expansion is for. I'm planning to replace my 8x250GB array with 3x1.5TB array on an 8 (maybe 12) port controller. When I fill up that 3TB, I'll add another 1.5TB drive. But yes, WHS's Pool solution provides more options for expansion.

I'm not worried about 20W or even 200W.

I wasn't either until I really looked at my electric bill and what was contributing.

I can run 9 SATAII drives right off my mobo, no added controllers needed and that doesn't include my 10 USB ports and 2 FW ports that can all be used for the WHS pool and even if I want to add more SATAII ports I can get an 8 port card for ~240.00. Again, when you fill up your 8 drive array, you need to buy another controller and 3+ more drives, I can just trade my smaller drives for newer larger drives and they don't have to match or I can add more. Right now I can only get 1.5T drives but what happens in X-months when we can get a 4T drive? I can just add 1 or 2 when I need more space.

You can do that with good controllers to, you can swap out a small drive for a larger one and create a second array on the unused space of the large drive. Not quite as simple as the pool, but still workable.

Depending on the amount of data, it can and when you factor in the ease of expandability, I still think WHS is better.

That's perfectly valid, but for me using a WHS Pool with duplication would cost me significantly more in the long run due to the energy and drive costs.

[QUOTE]I'm never worried about running out of space or expandability. By time I run out of space, there will be bigger drives that I can add or swap for a smaller one. Parity would be nice but it's not a deal breaker for my WHS, which is used for more than just Sage. I enable redundancy for my critical files but not every file so my total storage use is better than 50%[QUOTE]

Everything I put on my "solution" I want redundant. So requiring 14 discs to achieve redundancy on 7 makes it a dealbreaker for me.

I've been an IT professional for 14 years and I got my first computer in 1982, I understand RAID and I've seen RAID arrays that failed and were not recoverable short of restoring from tape. I'd say nice try, but it wasn't.

OK, let's just confirm some simple facts then, since your responses above were at least confusing:
RAID-5

1. Sacrifices 1/n drives worth of space for redundancy, leaving you with a useable capacity of (n-1)*min(size of drives)
2. When you write x amount of data to the RAID-5 array, it consumes x amount of the usable capacity from 1.
3. Can survive a single drive failure
4. Provides redundancy for everything written to the array and a best=worst storage efficiency of (n-1)/n

In comparison Windows Home Server's Drive Pool Folder duplication

1. Provides you with a useable capacity of n*size
2. When you write x amount of data do a duplicated folder, it consumes 2*x amount of the useable capacity from 1.
3. When you write x amount of data to an unduplicated folder, it consumes x amount of the useable capacity from 1.
4. Data in duplicated folders can survive a single disk failure, data not duplicated would be lost.
5. Provides redundancy for only data writen to duplicated folders
6. Provides best case 100% storage efficiency with no redundancy
7. Provides worst case 50% storage efficiency with full redundancy

Do you agree with those statements?

[QUOTE]I don't use external drives in my pool either but it is an option that RAID can't offer.[QUOTE]

You can use external devices with eSATA and SAS.

With the latest PP1, you can also use external drives to backup/archive unused/seldom used/static data.

You can do the same with a RAID-5 array. Again, my statements aren't a comparison of a RAID-5 array vs a Windows Home Server system, it's a comparison of a RAID-5 array and WHS's folder duplication as a solution to a requirement for redundancy.

1100+ for the controller and 19 drives. Which saves me 1100, IF I were willing to use FW/USB for the pool.

Except if you (like the OP and I) want all that data redundant, the $1100 will almost double your usable space vs folder duplication. You also ignored the fact that you can match the usable capacity of that 19 drive, fully redundant pool for $300-400 less with a good RAID card and only 10 drives.

Which is exactly why I said, in my first post in this thread: "I would suggest WHS as the OS instead of using a RAID array" (on XP-64) as the OP had posted.

You did read my response right, I said specifically that folder duplication was far less efficient, and not cost effective for large amounts of data. I didn't say anything about using (or not) WHS. The OP wants all data on redundant storage, so I simply stated that the OP needs to consider that WHS is not very efficient in that situation.

You'd be surprised how many IT professionals don't do proper backups on their home networks.

FWIW I keep all my irreplaceable data on at least two machines plus DVD backups.

[S_M_E]

Originally Posted by stanger89

You might want to use more drives, but you don't need to. You can create a RAID-5 array of 24 drives and only one drive's space will be used for parity information.

It would depend on the controller and backplane, older Sun and Dell 14 drive arrays with dual channel SCSI controllers could be configured to use all 14 drives on one channel and therefore use 1 drive for parity but you sacrifice performance. By using both channels you get better performance but you can only use 1/2 of the drives on each array, therefore requiring more drives for parity.

Yes, but look at the OP. The OP wants a redundant storage solution: "I also need to build some sort of redundant array". Hence we're talking about redundant storage solutions.

WHS can provide redundancy, as we've discussed , which again, is why I suggested using WHS for the OS instead of XP-64 with an array. I still stand by that suggestion.

If you just want a bunch of drives to show up as one location and don't care about redunancy, then there's no point going RAID in the first place. Hence, like I said I'm comparing Windows Home Server and RAID-5 in regards to their solution/implementation of redundancy (which is what the OP asked about). I'm not comparing RAID-5 to an operating system.

I'm saying that WHS is the best of both worlds, it CAN provide redundancy AND ease of use. It may not be the most efficient for the biggest collections but the good outweighs the bad by far, imo.

No, but that's the point, you go RAID-5 because the cost of mirroring is prohibitive.

That's one reason but not the only reason. There's also a performance difference. Performance isn't an issue with my Sage/WHS usage.

We do intend to fill our storage right? If so, then WHS's folder duplication does not hold a capacity advantage over RAID-5, quite the opposite.

No, not full. I prefer to keep some free space and if I get "too full" I can add more drives or exchange smaller drives for larger ones without replacing 3+ at a time.

That's what Online Capacity Expansion is for. I'm planning to replace my 8x250GB array with 3x1.5TB array on an 8 (maybe 12) port controller. When I fill up that 3TB, I'll add another 1.5TB drive. But yes, WHS's Pool solution provides more options for expansion.

Some controllers have features that others don't and they usually charge extra for those features. While you might be able to dynamically add more 1.5T drives (until you run out of ports) with some controllers, I can add any size drive, without matching drive sizes.

I wasn't either until I really looked at my electric bill and what was contributing.

My home LAN runs 8 boxes 24/7, 200W means nothing to me and cost isn't an issue.

Not quite as simple as the pool, but still workable.

Simplicity has it's advantages.

Everything I put on my "solution" I want redundant. So requiring 14 discs to achieve redundancy on 7 makes it a dealbreaker for me.

I'm not telling you (or anyone else) how to run your system, I suggested that the OP consider WHS instead of XP-64 and RAID.

OK, let's just confirm some simple facts then, since your responses above were at least confusing:
RAID-5

1. Sacrifices 1/n drives worth of space for redundancy, leaving you with a useable capacity of (n-1)*min(size of drives)
2. When you write x amount of data to the RAID-5 array, it consumes x amount of the usable capacity from 1.
3. Can survive a single drive failure
4. Provides redundancy for everything written to the array and a best=worst storage efficiency of (n-1)/n

In comparison Windows Home Server's Drive Pool Folder duplication

1. Provides you with a useable capacity of n*size
2. When you write x amount of data do a duplicated folder, it consumes 2*x amount of the useable capacity from 1.
3. When you write x amount of data to an unduplicated folder, it consumes x amount of the useable capacity from 1.
4. Data in duplicated folders can survive a single disk failure, data not duplicated would be lost.
5. Provides redundancy for only data writen to duplicated folders
6. Provides best case 100% storage efficiency with no redundancy
7. Provides worst case 50% storage efficiency with full redundancy

Do you agree with those statements?

Mostly, but depending on the configuration and timing, WHS could possible withstand more than a single drive failure.

You can use external devices with eSATA and SAS.

Not USB/FW, which is what most people have when it comes to external enclosures.

Again, my statements aren't a comparison of a RAID-5 array vs a Windows Home Server system, it's a comparison of a RAID-5 array and WHS's folder duplication as a solution to a requirement for redundancy.

Does WHS provide the option of redundancy? Of course it does. If the requirement is redundancy, WHS can be the solution. Case closed.

Except if you (like the OP and I) want all that data redundant, the $1100 will almost double your usable space vs folder duplication. You also ignored the fact that you can match the usable capacity of that 19 drive, fully redundant pool for $300-400 less with a good RAID card and only 10 drives.

I've said, several times, that with larger data sets WHS loses an advantage in efficiency but it's still an option for redundant storage and it provides more than JUST redundancy. There's more to WHS than just folder duplication.

I think we're done now...

[stanger89]

Originally Posted by S_M_E

It would depend on the controller and backplane, older Sun and Dell 14 drive arrays with dual channel SCSI controllers could be configured to use all 14 drives on one channel and therefore use 1 drive for parity but you sacrifice performance. By using both channels you get better performance but you can only use 1/2 of the drives on each array, therefore requiring more drives for parity.

You obviously haven't noticed any of the recent SATA/SAS controllers. 3ware, Areca, Adaptec, all have controllers with up to 24 SATA-II/SAS connections, thus up to 24-drive RAID arrays using only 1 (or 2 for RAID-6) for parity, and they all scale with performance as the number of drives increase. Some controllers even support single arrays across controllers (not that I'd try it).

I'm saying that WHS is the best of both worlds, it CAN provide redundancy AND ease of use. It may not be the most efficient for the biggest collections but the good outweighs the bad by far, imo.

Fair enough.

That's one reason but not the only reason. There's also a performance difference. Performance isn't an issue with my Sage/WHS usage.

Yes there is a performance difference, RAID-5 (on a good controller) may be faster, they can easilly hit several hundred MB/sec transfer rates.

My home LAN runs 8 boxes 24/7, 200W means nothing to me and cost isn't an issue.

Ouch, I had 3 PCs running 24/7 and turning one off when not in use was a significant difference. 200W is 163KWh/month, that's about $16/month, where electricity is cheap.

Simplicity has it's advantages.

Agreed, as a principal at least.

Mostly, but depending on the configuration and timing, WHS could possible withstand more than a single drive failure.

It's generally not a good idea to to plan for "could", especially not when talking redundancy.

Not USB/FW, which is what most people have when it comes to external enclosures.

Using those in any sort of array/pool is asking for trouble IMO.

Does WHS provide the option of redundancy? Of course it does. If the requirement is redundancy, WHS can be the solution. Case closed.

I've said, several times, that with larger data sets WHS loses an advantage in efficiency but it's still an option for redundant storage and it provides more than JUST redundancy. There's more to WHS than just folder duplication.

I think we're done now...

My only disagreement was your inital asertion that data gets doubled when written to a RAID array in addition to the space used by the RAID redundancy, which is not correct.

[davephan]

Originally Posted by CarlR

Any complete backup solution needs to include offside storage. I use Carbonite, but there are several others out there that will let you store an unlimited amount of data for a reasonable fee ($50/yr). I'm a little concerned about security, so I encrypt the files locally (to a 2nd drive) and send the encrypted versions to the internet backup. The locally encrypted files also serve as a local backup mirror. I'm only talking about 100G or so of data, so it's just not worth setting up a RAID for that - it's easier to buy 2 drives and do a mirror.

The offsite backup through an Internet connection might make sense if you don't have to backup too much data, or can backup only the changes after the initial backup. The problem is Internet providers are limiting the amount of traffic per month. Comcast is finally admitting their limit is 250 gigs per month. Before they admitted that, Comcast would cancel your service if you went over their limit that they would not tell you. There would not be a way to find out how much of the 250 gigs per month you are using, so you could limit your traffic. In January, there is supposed to be a bandwidth meter coming out. Other Internet providers might also go down the same road. They like to charge you a lot, but prefer to have customers that rarely use the Internet that are paying the high monthly fee.

Dave

[CyRex]

Originally Posted by stanger89

Has the FlexRAID dev returned from the abyss? There doesn't appear to have been any development since May.

Looks like the final release of FlexRaid Basic 1.0 will be here is a few days...

-Dan

[Skirge01]

I'm working on installing it this week and will uprade to the latest version when it's released. There's a great writeup on their forum here about how best to use to work with WHS. I just finished converting all my storage drives to 64k clusters (thanks to this post from S_M_E), so I'm finally ready to begin using FlexRAID.

[S_M_E]

Glad my tutorial helped, let us know how that flexraid works out for you, I'll defintately wait on that one. I see no reason for it, it sounds like a waste of a drive to me. YMMV

[Skirge01]

I don't want this to turn into a pros-cons of RAID discussion, since those can get heated with some people, but why would you say it's a waste to protect yourself against a hard drive failure by using parity? I'm using this for the larger files I don't want to use duplication on, such as my ripped MP3s, DVDs, and Blu-ray/HD-DVD movies, since that would truly waste a LOT of drive space. "Losing" a single drive in order to protect several terabytes of data doesn't seem like a waste to me. For critical things that I can't replace, such as photos, tax returns, business documents, etc, I'll be using a true backup solution in the form of WHS' duplication.

At any rate, the final release was... umm... released... today. I'm downloading it now and will be trying to set it up over the next few days. I'll certainly report back. My plan is to copy a bunch of data onto the WHS shares, hoping that it spreads across at least two drives and then pull one of them, format it in another computer, and try to rebuild the data from the parity.

I'm not really utilizing WHS for anything except nightly computer backups, since I'm still getting various things configured (i.e. Sage, my tuners, cable routing, etc.) in the server. So, I'm risking the loss of absolutely nothing during my tests.

[S_M_E]

Originally Posted by Skirge01

I don't want this to turn into a pros-cons of RAID discussion, since those can get heated with some people, but why would you say it's a waste to protect yourself against a hard drive failure by using parity? I'm using this for the larger files I don't want to use duplication on, such as my ripped MP3s, DVDs, and Blu-ray/HD-DVD movies, since that would truly waste a LOT of drive space. "Losing" a single drive in order to protect several terabytes of data doesn't seem like a waste to me. For critical things that I can't replace, such as photos, tax returns, business documents, etc, I'll be using a true backup solution in the form of WHS' duplication.

At any rate, the final release was... umm... released... today. I'm downloading it now and will be trying to set it up over the next few days. I'll certainly report back. My plan is to copy a bunch of data onto the WHS shares, hoping that it spreads across at least two drives and then pull one of them, format it in another computer, and try to rebuild the data from the parity.

I'm not really utilizing WHS for anything except nightly computer backups, since I'm still getting various things configured (i.e. Sage, my tuners, cable routing, etc.) in the server. So, I'm risking the loss of absolutely nothing during my tests.

We've already had that "RAID vs pool" discussion in this very thread, I use duplication for my music and ripped DVDs (etc), I do not use it for my recording share. I'd prefer to add the parity drive to my pool than use it for RAID. I've had issues with RAID on WHS in the past, I'll skip it. YMMV...

[stanger89]

Originally Posted by S_M_E

I'd prefer to add the parity drive to my pool than use it for RAID.

FlexRAID is a parity drive (or folder) not "RAID" per se.

[Skirge01]

Wow... you're right. I remembered the messages, but had forgotten it happened in this very thread! ;D

Anyway, like you, I've had issues with proprietary RAID in the past. That's what I like about FlexRAID: it doesn't... encode (for lack of a better word)... the data so that only the RAID software can read it. You can swap out a drive and still access the data in another computer. Since duplication is not cost effective in every scenario, FlexRAID seems to fill that gap nicely. We'll see how it turns out.

[S_M_E]

Originally Posted by stanger89

FlexRAID is a parity drive (or folder) not "RAID" per se.

Back to semantics I see. The effect appears to be the same.

Originally Posted by Skirge01

Wow... you're right. I remembered the messages, but had forgotten it happened in this very thread! ;D

Anyway, like you, I've had issues with proprietary RAID in the past. That's what I like about FlexRAID: it doesn't... encode (for lack of a better word)... the data so that only the RAID software can read it. You can swap out a drive and still access the data in another computer. Since duplication is not cost effective in every scenario, FlexRAID seems to fill that gap nicely. We'll see how it turns out.

The openegg thread you posted seem to suggest that WHS works well with it but it's just a matter of preference. I prefer to use the pool, others don't. I could change my mind, in the future, but I'm happy with the way I have things working.

As you mentioned, you're not going to lose data to your test machine so, by all means, give it a shot. If I hadn't experimented with unsupported workarounds I wouldn't have the system I have now. I wouldn't be able to change the strong password requirements, I would be stuck with a 20G C partition and I'd still have 4K clusters in the pool. I have 6 WHS tutorials, in the WHS forums for doing different things, most (if not all) are unsupported.

Like I said, let us know how it works out...

[stanger89]

Originally Posted by S_M_E

Back to semantics I see. The effect appears to be the same.

It's actually very different, the only thing it has in common with RAID, is the parity used for redundancy.

It just uses normal drives, with normal formatting, and then computes parity on those drives/folders and writes the resulting parity to a spare disk (as a file).

It's got all the benefits of the Pool you like, arbitrarilly sized disks, no special formatting, but with the redundancy efficiency of RAID-5.

[CyRex]

Originally Posted by stanger89

It's got all the benefits of the Pool you like, arbitrarilly sized disks, no special formatting, but with the redundancy efficiency of RAID-5.

From what I remember - correct me if I'm wrong - the parity 'drive' doesn't even need to be on the same computer, it can exist anywhere you want...

-Dan

[Taddeusz]

Sure, it's semantics, but if it's not RAID they shouldn't call it RAID. And it's not by definition any form of RAID type.

[S_M_E]

At least you admit that it's semantics.

I think we all understand that it's not true RAID and that it's actually RAID-ish but as far as I'm concerned it's not needed, for my situation, and I don't plan on using it, especially right away. I have plenty of free space and room to add more space if it should get low. I just see it as superfluous.

[stanger89]

Originally Posted by S_M_E

Back to semantics I see. The effect appears to be the same.

You stated you'd rather add a parity drive to your pool than convert to a traditional RAID array, implying that FlexRAID works like a traditional RAID array, thus you'd be uninterested in it. So I pointed out that it works like you'd like, it works on a normal group of drives and writes the parity to a separate drive, ie it's exactly like you'd prefer, pool + parity drive.

And you fall back to "it's semantics".

[Skirge01]

The way I see it:

A. We're wasting drives for parity.
B. You're wasting drives for duplication.

Same effect, different cause.

What a waste...

[Taddeusz]

Originally Posted by Skirge01

The way I see it:

A. We're wasting drives for parity.
B. You're wasting drives for duplication.

Same effect, different cause.

What a waste...

At least with RAID 5 parity is interleaved over all the drives. Drives can't necessarily be wasted on parity. Because if one of the drives fails that data can be recreated. If you're on a simple striped array with no parity or duplication you've lost everything unless you have a recent backup.