Moving to unRAID

[benjamintm]

Hi All,
I've decided to move my SageTV setup from Windows to an unRAID setup. This means I have to reconfigure my system significantly and do a bunch of moving of bits around to free up things so I wanted to get some advice.

Here's my current setup for Windows:
Motherboard: Intel DH67CF
CPU: i5-2400S @ 2.50Ghz
PSU: 340 watt bronze
Memory: 8 (4GBx2) DDR3
OS SDD: Samsung 840 SSD - 240GB
Media HDD: Seagate ST4000DM000 4TB
SageTV Recordings HDD: WD Black WD1002FAEX 1TB
Tuners: 4x HDHomeRuns Duals (all pre-2011 models)

After I migrate to unRAID, it will be identical except I will remove the SSD and re-arrange and add hard drives like this:
Parity Drive: Seagate ST4000DM000 4TB (from above)
Array Drive 1: WD Green WD20EARS 2TB (from a Drobo)
Array Drive 2: WD Green WD20EARX 2TB (from a Drobo)
Cache Drive: WD Black WD1002FAEX 1TB (from above)

I'll also be adding a SYBA SI-PEX40094 PCIe 2-Port SATA III adapter as my motherboard only has 4 SATA ports (only 2 at 6.0 Gb/s) and I want to add additional drives at some point in the future.

My plan is to use the array and parity for media and backups; and the cache as my storage for recordings and docker containers.

I've spent the last month prepping for this, copying the media drive over to the 2x2TB unRAID array and this weekend backing up my recordings to a couple of 1 TB drives I had from a Drobo. Everything is in place and I'd like to migrate the weekend of March 11, but I want to get feedback from the community to some questions I have.

1. Does anyone see any issues with the above re-configuration?
2. With the SageTV Docker, if I have media on the array, will unRAID be able to shut down the array and cache drives when not in use?
3. Can I just copy the wiz.bin file over to the unRAID SageTV docker and it will just read as is?
4. I want to use PlexTV as an iPad interface for recordings. I can use BMT to convert the files to a recognized format, but can it also add the season and episode to the name without mucking up SageTV mapping the recording to the file?
5. While I'm not planning on using the Samsung 840 SSD, if I understand unRAIDs cache drive mechanics, I could use it as my cache drive and the move recordings to the array at a regular basis (via mover). Has anyone done this? Is there any impact to performance on the SageTV extenders when bringing up recordings?

Your help is appreciated.

Thanks,
Ben

[Fuzzy]

Originally Posted by benjamintm

[*]Does anyone see any issues with the above re-configuration?

No, that hardware seems fine. If the Black drive is working for your full recording destination now, it should continue to do so on unRAID as the cache drive as well.

Originally Posted by benjamintm

[*]With the SageTV Docker, if I have media on the array, will unRAID be able to shut down the array and cache drives when not in use?

unRAID will spin down any drive not being used. default timeout is half hour I think, but it's configurable.

Originally Posted by benjamintm

[*]Can I just copy the wiz.bin file over to the unRAID SageTV docker and it will just read as is?

yes, sage will scan the configured recording and import folders, and re-link the files to the database entries. As long as the filenames are unchanged, it doesn't care about them moving to different folders.

Originally Posted by benjamintm

[*]I want to use PlexTV as an iPad interface for recordings. I can use BMT to convert the files to a recognized format, but can it also add the season and episode to the name without mucking up SageTV mapping the recording to the file?

Instead of renaming, probably best to use the SageTV Plugin for plex, while will pull in the metadata while using the files as they are.

Originally Posted by benjamintm

[*]While I'm not planning on using the Samsung 840 SSD, if I understand unRAIDs cache drive mechanics, I could use it as my cache drive and the move recordings to the array at a regular basis (via mover). Has anyone done this? Is there any impact to performance on the SageTV extenders when bringing up recordings?[/LIST]

I am recording to a cached share, and it seems to work fine. Gets written to the SSD cache drive during recording, and the 'mover' runs on a schedule and moves the content onto the array.

[stuckless]

My only comments is that I don't see the point in using a 1TB regular drive for cache, and I'd just move it into the Array. Typically your cache drive would be one or more SSD drives and you'd use them for disk IO performance.

[Fuzzy]

Originally Posted by stuckless

My only comments is that I don't see the point in using a 1TB regular drive for cache, and I'd just move it into the Array. Typically your cache drive would be one or more SSD drives and you'd use them for disk IO performance.

Well, I do think a single 1TB WD Black is going to have FAR better write performance than parity written array of WD Greens.

Writing a block to the cache drive takes the time it takes to seek to that position, and write that block.

Writing that same block to the array takes the time it takes to seek to that position on the drive, read that block, seek back to that position and write the block while also seeking to that position on the parity drive, reading the parity block, seeking back to the position and writing the modified parity for that block. Yes, and SSD is going to be faster than the WD Black, but most of the speed improvement will come from the direct write than the type of drive it's going to.

unRAID's turbo write feature is a lot quicker than the above, if enabled, but requires all drives to be up and spinning for any write to the array. It essentially reads that block from every drive, and calculates new parity to write to the parity block while writing the data block.

[wayner]

I am using an SSD as a cache drive on unRAID to hold my appdata partition.

I haven't done an A/B comparison on unRAID but I know that when I switched my Windows SageTV server system drive to an SSD many years ago it hugely improved performance of my system.

Wouldn't this also be the case with unRAID? Especially if you have lots of Fanart that has to be read in when displaying screens?

[Fuzzy]

Yes, random access of many smaller files is greatly improved with an SSD, because seek times are essentially zero. That doesn't mean appdata/sagetv HAS to reside on an SSD, and even if it does, it doesn't HAVE to be an unRAID cache drive. You could just as easily mount an SSD as just a bare drive, and point the docker to store stuff there. The only real reason appdata defaults to the cache drive is because it is expected to the be fastest location on a 'basic' unRAID server.

[BobPhoenix]

I would agree with Fuzzy. If you only have a one or two simultaneous recordings it would probably be fine going directly to the array. In my case I've had 10 going at a time for the length of the padding on the respective recordings - usually about 4 minutes. I have it going to 4 separate recording drives assigned with the "Unassigned Devices" plugin so that I can keep the number of recordings going to an individual drive lower during those peak times.

[panteragstk]

I record directly to individual drives within my "recorded tv" share and it works great.

I will say that I'd check those green drives to make sure they are still OK. I've had quite a few of those and only two are still being used. I have a first gen black 1tb that is still going strong along with 1tb green that is also doing well. For some reason the 2tb green drives were not that good for me.

Just want to make sure you don't have any issues going forward.

[benjamintm]

Originally Posted by BobPhoenix

I would agree with Fuzzy. If you only have a one or two simultaneous recordings it would probably be fine going directly to the array.

I've had 8 simultaneous shows recording from the HDHomeRuns two or three times. My wife and I just record anything that looks mildly interesting and then swing back to it at a point in the future. The WD Black has held up well so far. I'm planning on replacing it with dual SSDs in the future.

The 2TB WD Greens are older and I will be upgrading them to proper Reds (or equivalent) at a point in the future, but hopefully they should last. There isn't anything on them that can't be easily recovered.

Thanks for the feedback. Now I just need to take the plunge.

Ben

[BobPhoenix]

Originally Posted by benjamintm

I've had 8 simultaneous shows recording from the HDHomeRuns two or three times. My wife and I just record anything that looks mildly interesting and then swing back to it at a point in the future. The WD Black has held up well so far. I'm planning on replacing it with dual SSDs in the future.

The 2TB WD Greens are older and I will be upgrading them to proper Reds (or equivalent) at a point in the future, but hopefully they should last. There isn't anything on them that can't be easily recovered.

Thanks for the feedback. Now I just need to take the plunge.

Ben

Right. Going to a 7200 Black I could see 8 at a time. But 8 to the unRAID array where each write will take 4 IO operations each not so much. So say they get split equally to the data drives it would be 8 to each data drive and 16 to the parity drive. That might be too much for it especially considering the data drives are WD Greens and one is an older model EARS. The Seagate is 5900 I believe so a little slower and it would likely have problems keeping up with 16 IO operations for the 8 recordings. Going to the black as cache I wouldn't wont to chance it personally but since you have had good luck that way I see no problems with it as cache drive.

[benjamintm]

Originally Posted by BobPhoenix

Right. Going to a 7200 Black I could see 8 at a time. But 8 to the unRAID array where each write will take 4 IO operations each not so much. So say they get split equally to the data drives it would be 8 to each data drive and 16 to the parity drive. That might be too much for it especially considering the data drives are WD Greens and one is an older model EARS. The Seagate is 5900 I believe so a little slower and it would likely have problems keeping up with 16 IO operations for the 8 recordings. Going to the black as cache I wouldn't wont to chance it personally but since you have had good luck that way I see no problems with it as cache drive.

Pretty much my thought. I never tried to used the Greens for recording. At the time I bought it, the Black was the best performing, reasonably priced drive.

Of course right after I switch to unRAID my WD Black will die, laughing at me in it's whirl-clicky way.

Ben

[EnterNoEscape]

Originally Posted by benjamintm

Pretty much my thought. I never tried to used the Greens for recording. At the time I bought it, the Black was the best performing, reasonably priced drive.

Of course right after I switch to unRAID my WD Black will die, laughing at me in it's whirl-clicky way.

Ben

I've balanced between two 1.5TB WD greens for recording. I don't recommend it. It doesn't take much activity to end up with a few bad recordings.

[benjamintm]

After some delay (pesky daylights savings), I had an opportunity to migrate my server to unRAID. The process was a bit time consuming, but in the end everything is working with one minor issue.

I'd figure I'd give an overall summary of what I did.

Previously I had all of my ripped data on a 4TB Seagate ST4000DM000. I needed to move this data off as the drive was going to become the parity for the system. To do this, I pulled out two 2TB WD greens I had from an unused Drobo, booted into unRAID, and created an array out of the two drives. I then used rsync to move all of the ripped media from the 4TB to the array. This took an unusually long time, which I later realizes was a result of using rsync instead of the cp. rsync is great for incremental transfers of data, but cp is significantly faster when you just need to move a bunch of data. I wasted a lot of time waiting of rsync.

After the ripped data was copied over, I needed to copy my recordings over, since I was going to continue to use my 1TB WD Black WD Black WD1002FAEX for recordings. I did this in Windows with a pair of older 1TB hard drives (from the Drobo) connected via USB. I used rsync this time because I moved the data every night over a week so the new shows recorded were copied.

Then last Saturday I pulled open the computer, connected up the drives and booted unRAID. This is were I ran into a problem. For some reason unRAID refused to mount my WD1002FAEX Black as a cache drive. It would see it, allow it to be assigned, but kept saying it was unmountable, even after I told unRAID to format the drive. After a bit of futzing about, I ssh'd in and fdisked the drive and then formatted it with xfs. This allowed unRAID to mount the drive.

I then installed the SageTV docker containter and configured it to use the WD Black drive to store recording and started up my extender. And my goodness it was slow. Every action seemed to give me a spinning circle and remote key presses got queued up. After some more tinkering, I realized that the appdata was on the array and this was slowing things down. Once I moved the appdata to the WD Black, everything sped up nicely.

It's been almost 5 days and there haven't any issues. I've added some additional dockers (crashplan and MineOS). But the primary purpose of the server is to serve media and with the SageTV docker container it's working great.

Thanks to everyone for the feedback and help. Thanks to the community for continuing to develop SageTV. And thanks to Stuckless for his work on the docker container.

Ben