WHS - To Pool or Not To Pool...

[matterw]

I am putting together a new server with a WHS O/S. There are a number of threads (e.g., like this great one http://forums.sagetv.com/forums/showthread.php?t=37483) and have seen mixed messages about using the WHS Drive Extender (aka, storage pooling) for SageTV Recordings, specifically.

If you have experience using pooling, good or bad, I'd like to hear about it before I go in one direction or the other (btw, leaning towards pooling to take advantage of expansion).

Thanks,
Matt

[JetreL]

I currently use WHS with 2 HD-PVRs and 1 Silicondust HDhomerun recording 2 OTA streams with 4 drives pooled. There have been times when the system could not keep up with DiskIO. This on a quad-core system with 4 gigs of ram.

I believe Sage does a good job with drive management and would suggest keeping some drives separate for SageTV recordings. (This may not work for you but, I could care less if I lost all my recorded TV shows because those are pretty easily recovered)

I am currently working to remap my drive structure so that Sage records its files to its own drives and keep my DVD rips, images, and music on the clustered drives with duplication on, so I do not lose that data given the hours it would take for me to recover.

WHS is fairly modular. You can add and remove drives from the clustered drives pretty easily. (Depending how full the drive is dictates how long it takes to remove the drive) So you can play around to find whatever works best for you.

I hope this helps.

[gplasky]

I use pooled drives. All except the system OS drive are in the pool and formatted to 64k. I do NOT do file duplication. I don't care if I lose TV recordings and I own the DVDs and CDs that are ripped. Photos are also on another PC (as well as music). I have 4 drives varying from 500 GB to 1TB and to 1.5 TB. I add drives as I need them and just connect a 500GB USB drive and tell WHS I want to remove one of the internal 500GB drives. When the files are done moving I just pull out the old drive and stick in the new one. (Either a 1.5TB or 2 TB drive this time.) And then tell WHS I want to remove the external drive from the pool. All the files are transferred back and mostly to the new drive. (I'm usually at 80% full when I get a new drive so that most files transfer back to the new drive.) That is my main reason for using pooled drives since I don't do the folder duplication. I don't have any playback issues with the HD recordings. I'm not doing BlueRay yet so I haven't tested that. But it shouldn't be an issue. I have had 4 HD recordings at once while watching a HD recording on 2 extenders with no issues.

Gerry

[farscapesg1]

I use the pooled drive method... but I'm seriously reconsidering now that I started buying GP drives. I love the drive pool for the reason gplasky mentioned (easy to add space, replace drives, etc.) but I've been unhappy with the performance on my 15EADS drive I added to the drive pool. Of course, since it has the most space available everything is being recorded to it. Considering I run Showanalyzer as TV shows are being recorded, that is a lot of IO traffic on the drives. Running more than one instance of Showanalyzer and trying to watch the Olympics in HD w...a...s....s...t...u...tt...e...r...ing for example I think I've improved the performance by moving the 15EADS to the motherboard SATA port (instead of the Supermicro AOC-SATA-MV8 I'm using in a PCI slot), but that is going to cause problems as I add more drives...

I'm seriously considering picking up another 1.5TB drive to replace two 500 GB RE2s and move them out of the drive pool and do all my recording to them. Sorta defeats my purpose of replacing drives with GP ones to cut down on heat and power though Of course I'm also toying with the idea of replacing the SATA card with a AOC-SASLP-MV8 to take advantage of the higher bandwidth of the PCIE slot... I'm just worried about the GP drives still not being able to handle the IO from multiple recordings/showanalyzer/playback/etc.

[gplasky]

Originally Posted by farscapesg1

I use the pooled drive method... but I'm seriously reconsidering now that I started buying GP drives. I love the drive pool for the reason gplasky mentioned (easy to add space, replace drives, etc.) but I've been unhappy with the performance on my 15EADS drive I added to the drive pool. Of course, since it has the most space available everything is being recorded to it. Considering I run Showanalyzer as TV shows are being recorded, that is a lot of IO traffic on the drives. Running more than one instance of Showanalyzer and trying to watch the Olympics in HD w...a...s....s...t...u...tt...e...r...ing for example I think I've improved the performance by moving the 15EADS to the motherboard SATA port (instead of the Supermicro AOC-SATA-MV8 I'm using in a PCI slot), but that is going to cause problems as I add more drives...

I'm seriously considering picking up another 1.5TB drive to replace two 500 GB RE2s and move them out of the drive pool and do all my recording to them. Sorta defeats my purpose of replacing drives with GP ones to cut down on heat and power though Of course I'm also toying with the idea of replacing the SATA card with a AOC-SASLP-MV8 to take advantage of the higher bandwidth of the PCIE slot... I'm just worried about the GP drives still not being able to handle the IO from multiple recordings/showanalyzer/playback/etc.

Did you download and run the Drive Balancer utility I told you of in the other thread? Running this will move files among all the drives so that they have approx. the same amount of files on each one including your new one. This should fix your issue with the new drive being mostly empty and being used for all new recordings. Plus WHS will be trying to balance the files slowly in the background and adds to your issue.

Gerry

[toricred]

I'm currently using pooled drives. I have 1 x 1TB, 4 x 1.5TB, and 2 x 2TB. They're all formatted to 64K. I use 4 x HDPVR and 3 x HDHR (total of 10 HD tuners). For playback I have 3 x HD100 and 1 x HD200. I have SAV1 doing live detection up to 3 shows at once.

I've had problems in the past caused by one drive in the pool, but it turned out to be a combination of a firmware upgrade for the drive and a marginal SATA cable.

I use Drive Balancer once in a while, but after running it the first time my drives tend to stay more or less balanced. This configuration is working incredibly well as long as I watch out for problems with the drives themselves (like a drive dieing). I need to upgrade the 1TB drive to a 2TB drive soon, but other than that things are great.

[Skirge01]

I'm also running all of my drives in the drive pool and am quite content with the performance. I currently have 6 WD10EVCS drives and 4 Samsung HD103UJ drives, all in the pool and formatted to 64k. I have 2 HD-PVRs and 4 HDHRs for OTA, as well as run a max of 3 comskip instances on my quad core in the WHS box. Even maxed out, I have never had an issue with disk IO. If you want to see how everything is connected and all the specs, check out my blog linked in my signature.

After saying all that, I will add that I intend to steer clear of any "green" drives in the future because I don't see a benefit to using them. The power "savings" is minimal and may never actually be realized, so I don't see any cost savings ever coming from their usage and, hence, no benefit to the green drive's added cost.

[sic0048]

IMHO, the only reason to pool the drives in WHS is to utilize the duplication function. Personally I do not care to have my recorded shows duplicated. Therefore I do not run my recording drives in the WHS pool. I have these two drives outside the pool.

Sage does a fine job of handling the drives and basically acts as if the drives are pooled anyway. After all, Sage handles which drive it will record to, it scans the drives for media, and it is a seemless experience for the end user. They do not ever need to know the drive letter once it is inputed into the SageTV setup.

For all these reasons, I feel that putting recording drives into the WHS pool is at best a waste of my time and energy, and at worse a potential issue due to the heavy i/o. I'd rather not have these recordings tying up WHS resources.

[PLUCKYHD]

64k is a must but there is no issues with mine having recordedtv in the pool. I don't duplicate it but I want it in the pool so it has all the storage avaliable to it. I have recorded 4 HDTV feeds while running two instance of showanalyzer without any issues.

[Spartan]

I have 3 500GB Seagate drives in the pool. I have duplication active on some shares, but not the recordings directory. I have NOT formatted to 64k blocks because it seemed to risky to try and pull off.

I have a Sage Client & and a HD200, and only see stuttering when I move files (large files) while both clients are running and something is being recorded. This doesn't happen very often.

Normal use is one playback device and 2 HD recordings (perhaps an analog recording too), each with it's comskip and that works just fine.

[Beefcake550]

I have 3x1TB GP dives, 1x1.5TB GP, and 2x1.5TB Seagate drives. I pool all of them and formatted to 64k without issue. Simply follow the guide and it works. I agree that 64k cluster size is a must in order to have trouble free pooled drives.

My main source of data are movie rips and tv recordings. I have 1 HDPVR and 1 HDHR. I serve up 2 HD200s at once and a PCH-A110 thrown in for fun. Flawless operation with 64k cluster and as the author of Drive Balancer, I use that tool on a regular basis to move files around and even up the data utilization.

[Djc208]

I'm with SIC0048 on this one. When I set up my WHS server I just left the recording drives out of the pool. One they were already properly formatted and had recordings on it, so it was easier to add them outside the pool than to try and add them to the pool, and as SIC said Sage doesn't care or show drive letters and file paths.

Plus I felt it was just easier to manage that way. Those drives are for recordings only and Sage can use all of them as it needs to. The pool is for all my other storage with folder duplication for the important stuff (pictures, etc.).

When I finally decide to start ripping my DVDs/BD/HD DVDs to the server I will probably put that in the pool since those drives are easier to share across the network and the content is much more static.

[PLUCKYHD]

Originally Posted by Djc208

I'm with SIC0048 on this one. When I set up my WHS server I just left the recording drives out of the pool. One they were already properly formatted and had recordings on it, so it was easier to add them outside the pool than to try and add them to the pool, and as SIC said Sage doesn't care or show drive letters and file paths.

Yeah but to me that kind of ruins the purpose of WHS in the pool aspect. By adding the recording drive to the pool you get full access to the entire pool for recordings and if you every time you add more space everything gets access too it (videos,recordings,pictures etc). otherwise if you recording drive is outside the pool and gets full you either have to replace it with a bigger drive or add another drive outside your pool again. To me seems to ruin the purpose of the pool my that is my 2 cents to each his own.

[sic0048]

Originally Posted by PLUCKYHD

Yeah but to me that kind of ruins the purpose of WHS in the pool aspect. By adding the recording drive to the pool you get full access to the entire pool for recordings and if you every time you add more space everything gets access too it (videos,recordings,pictures etc). otherwise if you recording drive is outside the pool and gets full you either have to replace it with a bigger drive or add another drive outside your pool again. To me seems to ruin the purpose of the pool my that is my 2 cents to each his own.

I guess I can understand that arguement, especially if your machine has limited space/capacity for hard drives. If it difficult to add a new drive, but easy to replace an existing smaller drive, then I can understand the attraction to pooling those drives. Then you only have to unpool the small drive, wait for WHS to rebalance everything, and then replace the small drive with a large drive.

But I'm lucky enough to have built a relatively new machine that has 8 onboard SATA plugs. I currently have 6 drives hooked up. Two outside the pool dedicated to SageTV (1tb and 750gb) and four drives that I used for the WHS pool (2x 500gb drives (one is the OS drive), 1.5tb, and 1tb). That gives me about 3.5tb of pooled space and 1.75tb of recording capacity for Sage.

I do a pretty good job of deleting shows after I watch them and generally don't archive any shows (but I do have the entire Jericho series saved which takes up a little bit of room). I only recently added the 2nd SageTV drive to ensure enough space to record the Olympics. Since I don't put recorded shows on the pooled drives, my total used capacity of the 3.5tb WHS pooled drives is still pretty low too.

[bsquarewi]

An added benefit of using a non-Pooled drive for Sage Recordings (for me anyhow) is that I also use a single 750gb non-pooled drive (64k) to install any additional programs or utilities which I want running either as part of the Sage system or otherwise so as not to crowd up that tiny 20GB C:\ drive.

[Skirge01]

Originally Posted by bsquarewi

An added benefit of using a non-Pooled drive for Sage Recordings (for me anyhow) is that I also use a single 750gb non-pooled drive (64k) to install any additional programs or utilities which I want running either as part of the Sage system or otherwise so as not to crowd up that tiny 20GB C:\ drive.

What does that 750GB drive have to do with recordings? Oh... okay, I think I get it. You ALSO use that drive for recordings, so you have to manage the space yourself and ensure that you continue to have room for recordings and any programs you decide to install on it, right?

[Skirge01]

Originally Posted by sic0048

I guess I can understand that arguement, especially if your machine has limited space/capacity for hard drives. If it difficult to add a new drive, but easy to replace an existing smaller drive, then I can understand the attraction to pooling those drives. Then you only have to unpool the small drive, wait for WHS to rebalance everything, and then replace the small drive with a large drive.

But I'm lucky enough to have built a relatively new machine that has 8 onboard SATA plugs. I currently have 6 drives hooked up. Two outside the pool dedicated to SageTV (1tb and 750gb) and four drives that I used for the WHS pool (2x 500gb drives (one is the OS drive), 1.5tb, and 1tb). That gives me about 3.5tb of pooled space and 1.75tb of recording capacity for Sage.

I do a pretty good job of deleting shows after I watch them and generally don't archive any shows (but I do have the entire Jericho series saved which takes up a little bit of room). I only recently added the 2nd SageTV drive to ensure enough space to record the Olympics. Since I don't put recorded shows on the pooled drives, my total used capacity of the 3.5tb WHS pooled drives is still pretty low too.

Can't say that I agree. I have 10 drives as part of the pool, which are used for recordings, photos, music, DVDs, and general file storage (financial data, among everything else). Many of these folders have duplication turned on, except for the single recording directory I created. I have plenty of room in my case for more drives, but the ability to add and remove drives via WHS is so simple, it seems silly to manage them all myself. I don't want to know, nor do I care, which drives my files are stored on. I'm happy to let WHS figure all that out for me.

[PLUCKYHD]

Originally Posted by Skirge01

Can't say that I agree. I have 10 drives as part of the pool, which are used for recordings, photos, music, DVDs, and general file storage (financial data, among everything else). Many of these folders have duplication turned on, except for the single recording directory I created. I have plenty of room in my case for more drives, but the ability to add and remove drives via WHS is so simple, it seems silly to manage them all myself. I don't want to know, nor do I care, which drives my files are stored on. I'm happy to let WHS figure all that out for me.

Agreed I have over 16tb and still growing I don't want to worry what goes where.

[Djc208]

Originally Posted by PLUCKYHD

Yeah but to me that kind of ruins the purpose of WHS in the pool aspect. By adding the recording drive to the pool you get full access to the entire pool for recordings and if you every time you add more space everything gets access too it (videos,recordings,pictures etc). otherwise if you recording drive is outside the pool and gets full you either have to replace it with a bigger drive or add another drive outside your pool again. To me seems to ruin the purpose of the pool my that is my 2 cents to each his own.

True but the reverse also applies. If you set Sage to leave X GB free then if you rip a new DVD to the drive, or come back and upload 2GB of pictures (with duplication). Sage now needs to go delete 4GB worth of shows in order to meet the above space requirement. Hopefully they're shows you don't want.

Neither way is wrong, just shows the benefits and weaknesses of both options.

Maybe I treat my recordings differently, or just don't record as much stuff, but I don't need any more space for recordings. I have two Sage drives totaling about 1.75 TB. Of that probably 40% is stuff I've watched and have Sage set to just delete as more space is needed. The remaining 60% is still almost 400 SD and HD shows (both favorites and IRs) I have yet to watch, and a lot of the IR stuff are older series that play often enough deleting them isn't really a big deal.

My pool is much smaller but still more than sufficient for all my photos (with duplication), music, personal files, and archived recordings and the odd video file.

[bsquarewi]

Originally Posted by Skirge01

What does that 750GB drive have to do with recordings? Oh... okay, I think I get it. You ALSO use that drive for recordings, so you have to manage the space yourself and ensure that you continue to have room for recordings and any programs you decide to install on it, right?

Yeah, sorry about that. I meant to say that the NON-Pooled drive I use for Sage recordings I ALSO use to install programs on outside the system drive partition.

Each person has different priorities when it comes down to what is it you want your WHS to do. For me, SageTV is just 1 small part of what I use my WHS for and pretty much the only thing I record is Football or use Sage for pausing live TV so the disk space need is minimal. Just a single HVR2250 tuner, not one of the 6+ tuner types. In my drive pool, however, I have several TBs of family home video and photos and DVD/BluRay rips.