Media Storage Recommendations

[Fuzzy]

Originally Posted by Steve20A

Fuzzy,

I like that idea. But, can you point me to docs that describe how you "configure to use cache?" I don't see an obvious reference in the Sage V7 docs. Also, some document direction for the "mover" function.

Or is this all your own code?

Thanks.

Steve

It's not a sage option, it's an unRAID setting on each share.

[mgpaulus]

So, just out of curiousity, how large are your recording setups? I currently have (2) 5TB drives and 4 (4TB) drives in my sagetv setup for recordings, and a 120GB SSD for the system drive. I think I have somewhere around 20TB of recorded media (It's at home, and I'm not right now). I haven't worked in single share spaces that are 25TB+, so I'm not sure how file operations work in that environment. Is it fairly speedy? And what if you want to go to command-line and look for a file? That's a large quantity of files to be mucking through. Just wondering how large others setups are and if they have performance or other issues running under unRaid?

[BobPhoenix]

Originally Posted by mgpaulus

So, just out of curiousity, how large are your recording setups? I currently have (2) 5TB drives and 4 (4TB) drives in my sagetv setup for recordings, and a 120GB SSD for the system drive. I think I have somewhere around 20TB of recorded media (It's at home, and I'm not right now). I haven't worked in single share spaces that are 25TB+, so I'm not sure how file operations work in that environment. Is it fairly speedy? And what if you want to go to command-line and look for a file? That's a large quantity of files to be mucking through. Just wondering how large others setups are and if they have performance or other issues running under unRaid?

I have several unRAID servers. The smallest is about 45TB. The only performance slowdown is when the drives are spun down it takes a while for them to spin back up. Once the drive(s) are spinning accessing an individual file in a shared directory across multiple drives is very fast. It appears to be as fast to me as a single drive access. If the spin-up is a problem just set the spin down delay to a larger time and they won't spin down as often. Or turn off the spin down delay and have your drives spinning all the time. I let my drives spin down because it does cut down on electricity and probably prolongs their life.

[Zogg]

Originally Posted by Fuzzy

Allocation methods don't really matter much when dealing with SageTV, simply because sage aggregates everything into a singular database anyway, so what 'drive' it's on in the NAS is of no real consequence. I have a single import folder, and a single recording folder, both are configured to 'use cache'. This means writes of new files will go to the cache drive, and then be moved to the pool later (when the 'mover' runs). This does a nice job of allowing lots of different shows to be watched (read from the pool, which is very fast), while not impacting the writing of current recordings, that are written to the much faster SSD cache drive.

So you record to one folder, and then the 'import' folder you mention is configured in Media Center Import Directories? I guess I thought that was for items to show up in the 'Videos' menu.

I need to set up recordings on one drive and then move them to a NAS later, so this would work the same way?

[BobPhoenix]

Originally Posted by Zogg

I need to set up recordings on one drive and then move them to a NAS later, so this would work the same way?

That is how I have my SageTV servers setup. You could record to recording directories directly ON the NAS as well. That is the method most use when using the SageTV docker on unRAID I suspect. I prefer to record to separate drives mounted with the "Unassigned Devices" plugin on unRAID and then when those drives are getting FULL I move the files to my unRAID array drives in an Import folder in SageTV docker.

[Zogg]

Originally Posted by BobPhoenix

That is how I have my SageTV servers setup. You could record to recording directories directly ON the NAS as well. That is the method most use when using the SageTV docker on unRAID I suspect. I prefer to record to separate drives mounted with the "Unassigned Devices" plugin on unRAID and then when those drives are getting FULL I move the files to my unRAID array drives in an Import folder in SageTV docker.

I've been recording to a folder on my NAS (NAS4free system) but I'm seeing occasional errors where the file can't be opened, even though I see the file get created with 0 bytes. I lost a lot of recordings this week since I was too busy to pay attention, so I'll just record to a local drive from now on and then move later.

And to clarify, the import folder would be set up in what SageTV calls the Media Center Import Directories?

[BobPhoenix]

Originally Posted by Zogg

And to clarify, the import folder would be set up in what SageTV calls the Media Center Import Directories?

Yes.

When I move to my array I create folders for each series and move my recordings to those directories. The root folder is a shared folder on my unRAID systems that is defined as a Media Center Import Directory.

So:
I have:
\\unRAIDServer\TV Series\Castle\Castle-S01E03-HedgeFundHomeboys-18241257-0.mpg
as the path to a TV Series file which shows up in SageTV Recordings grouping "Castle"

[Zogg]

Got it, thanks so much!

[texneus]

Zogg, are you recording to a ZFS share by chance? How many disks in your array and in what configuration?

[Zogg]

Originally Posted by texneus

Zogg, are you recording to a ZFS share by chance? How many disks in your array and in what configuration?

Yes, I'm running a ZFS share. Five 4TB disks in RAIDZ2 configuration. 44% usage according to the web admin.

My SageTV server is a separate computer running Windows 7, and right now I don't have any VM's running on the NAS, only using it for storage.

[texneus]

Originally Posted by Zogg

Yes, I'm running a ZFS share. Five 4TB disks in RAIDZ2 configuration. 44% usage according to the web admin.

My SageTV server is a separate computer running Windows 7, and right now I don't have any VM's running on the NAS, only using it for storage.

Cool. At this time I'm suspicious that ZFS itself may be the root cause of recording issues I'm seeing (different issues than you're apparently having, but issues none-the-less so just looking for another datapoint). Just a theory that I'm currently testing and will know more in a few days.

You might see if spin up delays are causing SageTV to think your share is off line, assuming you let your drives spin down. I have a pair of 4TB WD reds that take a good 2-3ish seconds each to come back on line if they spin down. Once they are awake they're fine but the share is understandably non-responsive until both drives are up, about a 5 second delay total. Recording on Windows 7 and copying to the NAS will definitely work though, I did exactly that for several years.

[Zogg]

Originally Posted by texneus

Cool. At this time I'm suspicious that ZFS itself may be the root cause of recording issues I'm seeing (different issues than you're apparently having, but issues none-the-less so just looking for another datapoint). Just a theory that I'm currently testing and will know more in a few days.

You might see if spin up delays are causing SageTV to think your share is off line, assuming you let your drives spin down. I have a pair of 4TB WD reds that take a good 2-3ish seconds each to come back on line if they spin down. Once they are awake they're fine but the share is understandably non-responsive until both drives are up, about a 5 second delay total. Recording on Windows 7 and copying to the NAS will definitely work though, I did exactly that for several years.

I originally had my drives set up to spin down but I didn't like the delay so I turned it off. Honestly, I think it's harder on the drives over time to continually spin up and down anyway based on my experience with electric motors.

I'm not really sure what the problem is, just that for some reason SageTV seems to have a problem opening the file for recording. And it's not consistent so it makes it really hard to troubleshoot.

But I also had some other odd issues this past week such as a recording from a Prime that did work, but the video won't play in SageTV no matter which decoder I choose. Plays just fine in VLC, MPC-HC, etc. though. And my OTA tuners, original HDHomeRun's, just quit getting a TV signal all week, but as soon as I started checking on it they all started working perfectly again.

I'm hoping that writing to a local drive will fix this one issue, and then I can focus one at a time on the other ones.