WHS - To Pool or Not To Pool...

[Skirge01]

Originally Posted by Djc208

True but the reverse also applies. If you set Sage to leave X GB free then if you rip a new DVD to the drive, or come back and upload 2GB of pictures (with duplication). Sage now needs to go delete 4GB worth of shows in order to meet the above space requirement. Hopefully they're shows you don't want.

Neither way is wrong, just shows the benefits and weaknesses of both options.

Maybe I treat my recordings differently, or just don't record as much stuff, but I don't need any more space for recordings. I have two Sage drives totaling about 1.75 TB. Of that probably 40% is stuff I've watched and have Sage set to just delete as more space is needed. The remaining 60% is still almost 400 SD and HD shows (both favorites and IRs) I have yet to watch, and a lot of the IR stuff are older series that play often enough deleting them isn't really a big deal.

My pool is much smaller but still more than sufficient for all my photos (with duplication), music, personal files, and archived recordings and the odd video file.

You're probably right that the small amount of storage you have is most likely why it makes sense for you. I have hundreds of recordings spread across 10 drives. Our household does a lot of TV watching. We don't subscribe to Netflix, so when a movie comes on cable, we record it. It may be months before we actually watch it. If we went looking for it when we wanted to watch it, it might not be on cable anymore. Anything I don't want to be deleted, I tell SageTV not to delete, so I never have to worry about a show being deleted until I decide to. SageTV currently says I have 265 shows taking up 2.1TB ready to be auto deleted, if necessary. Most of those shows are from when I have "suggestions" turned on. I just never went in and deleted them when I turned it off. That's something else I didn't feel like taking the time to manage.

[matterw]

Originally Posted by Skirge01

If you want to see how everything is connected and all the specs, check out my blog linked in my signature.

In your blog (which BTW is a great source of detailed information) you mention that you are looking at FlexRaid. Based on your reply it seems that you are no longer using it. I am curious to know your experience with FlexRaid and your decision around this.

[matterw]

Originally Posted by gplasky

Did you download and run the Drive Balancer utility I told you of in the other thread? Running this will move files among all the drives so that they have approx. the same amount of files on each one including your new one.

Forgive me as I am new to the WHS Driver Extender technology, but why do you need Drive Balancer? In other words, shouldn't WHS be taking care of the balancing of the data across the pool or is it just going to fill up the drives one at a time?

[JetreL]

Originally Posted by matterw

Forgive me as I am new to the WHS Driver Extender technology, but why do you need Drive Balancer? In other words, shouldn't WHS be taking care of the balancing of the data across the pool or is it just going to fill up the drives one at a time?

You would think that but it doesn't, you will see a huge discrepancy in utilization especially when you add drives and remove drives. The Drive Balancer works great, BTW.

[Skirge01]

Originally Posted by matterw

In your blog (which BTW is a great source of detailed information)

Thanks. Always nice to hear.

you mention that you are looking at FlexRaid. Based on your reply it seems that you are no longer using it. I am curious to know your experience with FlexRaid and your decision around this.

I did use it for a while, but it just couldn't handle the constantly changing nature of my recordings directory. To be fair to FlexRAID, the developer clearly states that it isn't intended to work with such data, but rather with data that stays fairly static. At the time, I was still using the "suggestions" feature of SageTV, so my tuners were constantly recording and deleting shows. The disk IO was far too great a load when a sync of FlexRAID was running.

However, now that I'm no longer using the "suggestions" feature, I have considered going back to FlexRAID. It did work exactly as designed and I was very happy with the results. My intended usage at the time simply wasn't meant for FlexRAID.

[KarylFStein]

Originally Posted by matterw

I am putting together a new server with a WHS O/S. There are a number of threads (e.g., like this great one http://forums.sagetv.com/forums/showthread.php?t=37483) and have seen mixed messages about using the WHS Drive Extender (aka, storage pooling) for SageTV Recordings, specifically.

If you have experience using pooling, good or bad, I'd like to hear about it before I go in one direction or the other (btw, leaning towards pooling to take advantage of expansion).

Thanks,
Matt

I personally put all my drives in the pool, (64k formatted), with duplication on, (including recorded TV--availability is what I'm going for). I know I've had two shows recording with ShowAnalyzer running on them, and had two playbacks going all at the same time, (one playback being one of the shows being recorded and the other a DVD rip). The only issue I saw is when the auto-commercial-skip kicked in on the show I was watching, I had "the circle" for a second or two, (I'm guessing while it seeked to the right place), but no stutters in playback. (I was watching a 1080i OTA recording and didn't test REWs/FFWs, which I probably should have.) No complaints from the kids watching the DVD, so assume that was smooth as they'd complain .

My WHS system is a Q6600 CPU with 3G RAM. I don't use drive balancer and my only "tweak" was the 64k block size on the drives.

This is the only time I've seen "the circle" since I switched to WHS and why I looked at what stress the system was under at the time. I don't think there's usually more than one recording (+SA) and one playback going at the same time around here...we're pretty light on the recording and viewing. My *guess* is anything more than the 2 recordings with SA running + 2 viewing would cause bigger problems, but don't know.

[SWKerr]

I have 3x1.5TB and 1x1TB drives in a pool. I keep 1x1TB drive out of the pool for backups. Everything is formatted at 64k as recommended. The only things I have duplicated are Pictures, Music and Personal stuff. Movies and TV are not duplicated. Movies, Pictures, Music, Personal stuff and Sage configs are all copied weekly to the Backup drive. TV would be lost in my setup in the event of a failure.

I can do four at a time recording and watch two shows at the same time without a problem. The only time I seem to have problems is when I try to backup the Movies to the Backup drive while the HD-PVR is recording. (Not small files like Music or pictures, Just movies.) My fix is to schedule the sync at 3am. I don't use any green\slow drives in the pool.

It all works great just hope I never have to recover the system drive. When I went from the Demo to the real box I looked at the drives in the pool and they had the data on the disk in a way I could figure it out and find what I needed. This is important to me since I have lost RAID setups at home and work and really just depend on the backups. (RAID is really not recoverable if it gets hosed.)

What I would like to see is the ability to select where the recordings get put in my favorites setup. I could do this in BeyondTV. That way I could put the stuff I really like in a duplicated folder and leave everything else to chance. Most of the shows are stuff I know will be repeated or that I have access to on Hulu.

[Beefcake550]

Another reason for pooled drives + drive balancer runs ewvery so often....

It will "disperse" your video files among all the drives in your pool. This will not only give even wear and tear on the drives, but you will be able to handle more read/write streams as a whole system since your media is distributed. That was my main goal when I wrote "Drive Balancer".

I'm coming out with a new version that does some more cleanup work and fixes a few bugs I've found when not including the "D:" drive in the effort to balance. I recently swtiched to this method now that I have a really fast 150GB 10krpm raptor drive for the OS drive.

[SWKerr]

Originally Posted by Beefcake550

I have a really fast 150GB 10krpm raptor drive for the OS drive.

Does a fast OS drive make a real difference in performance in WHS. I was thinking most of the disk access would be in the pooled drives.

I was considering using a SSD for the boot disk in WHS since they are getting cheaper. I was thinking it would be more reliable and faster.

Been looking for a definitive post on the minimum size WHS would use as well. I have seen 80GB but have read a few post that seemed to indicates success on a 64GB SSD.

[Skirge01]

I also believe it does, to a certain extent. I'm planning to replace my 7200rpm drive with an SSD at some point because I tend to see the OS drive getting very active at times and a matching slowdown of certain events, such as remoting into the server. However, I've never had an issue where SageTV wouldn't play back a recording. Perhaps a couple of seconds of the circle before the menu would respond, but that's all. Definitely nothing bothersome. In fact, I think my DirecTV STBs responded slower sometimes.

80GB is the minimum that MS requires for the OS. 20GB get used for the OS partition and the rest is the D: drive where the tombstones will lie. The tombstones are only 4k files, so there's no risk there, but the D: drive is also used as overflow for the drive pool. If you run the risk of repeatedly filling your pool up, WHS may frequently be storing files on the D: drive until there's space available in the pool again and can move it over. If this happens, you may see a slowdown, as well as the system complaining about no disk space. I used a 750GB, but will drop down to a 128GB when I go to an SSD, due to cost. But, I have tons of space in the drive pool, so the D: drive never gets used.

[leagle]

I record to the pool and it has worked beautifully. I have a 200GB OS drive and two 1TB Samsung Ecogreen drives in the pool. Both pool drives are 64k. For tuners, I have 1 HDHR and 1 HD-PVR. Currently, all of the viewing is done via a HD200 (adding a second one shortly). Our DVD rips directory is currently around 800GB and our recordings directory is at 400GB (varies greatly week to week). We also store music, photos, and assorted other files on the same WHS server. Duplication is turned on for everything except the recordings and dvd rips. Every night at 4am, Synctoy echoes all of my shared WHS directories to a freenas box. The only thing I don't have completely covered is the WHS OS drive. My WHS bx also runs CrashPlan* (highly recommended!), so I am backing up my sage program directory and a few other things to another system, but I would feel better if I could just image the entire drive.

This set-up has been rock solid ever since I straightened out my HD-PVR issues (1.0.5.301 = happiness). As others have said, there are a few things that you can do to make this kind of set-up work well:

- Use 64k block sizes on all of your pool drives.

- Don't use your WHS machine for much more than serving files. It really isn't designed to do a bunch of other things, so don't force it. Home automation and other low overhead applications are fine.

- Don't use a bottom of the barrel processor in your WHS machine. You definitely don't need the latest and greatest, but I feel like a lot of folks skimp on the processor and memory for WHS boxes. My hardware is a few years old (see my sig) and it works great.

- Don't use drive duplication on your recording directory. I don't use it on my dvd rips directory either, but I know that some folks do without any problems.

Everyone has their own preferences, but I guess that I just don't see a reason to not use the pool for recordings. If it makes things easier to manage and there's no performance hit, then why not? The only thing I have to keep an eye on is my pool usage. If that gets too high, I just throw a new drive in the sata backplane and tell WHS to add it to the pool. Done.



* CrashPlan uses a non-pooled drive for backups because of limitations in the CrashPlan software.

[Evil_Attorney]

I just found this thread after posting this thread about my WHS performance issues: http://forums.sagetv.com/forums/showthread.php?t=50700

From what you guys are saying, it sounds like my performance issues are normal. I am not using 64k clusters and am recording to a duplicated share. Sage is much slower to start recording live TV and has a large pause between live shows (this didn't happen on my XP software RAID). If so, I am really disappointed in WHS. It's just not built for any kind of media streaming performance.

It sounds like I should take one drive out of the pool, format at 64k, and use it only for Sage recordings. I really wanted my recordings duplicated, like with RAID 1, so I would never have any data loss if a hard drive failed. But looks like that isn't possible.

[sic0048]

What SageTV version are you running - V6 or V7? Keep in mind that this thread was primarily written under V6. With V6, you still need to worry about using 64k blocks and you are going to get a pronounced pause when a new recordings starts on LiveTV.

If you are using version 7, both of those issues have been "corrected" and you should not have to use 64k blocks and you should not be seeing a long pause.