Media Storage Recommendations

[drvnbysound]

My wife and I bought a new [to us] house a few months ago. We've finally finished quite a bit of aesthetic remodeling, including taking out a 5-ft section of wall, and I'm soon to get into the attic and start pulling data, speaker, etc cable all over the place. Between now and then, I've got some time and am considering storage options...

My previous STV setup was run on a "simple" W7 x64 system with a myriad of hard drives; I believe it was (1) 4TB drive for DVD rips, (1) 2TB drive for BD rips (obviously wasn't ripping as many of those) and (2) 1TB drives for DVR storage. I plan on getting my HDHRs set back up with an OTA antenna, and will likely continue doing local rips for the foreseeable future. Having said that, I'm looking for recommendations on a "better" way to store them. I've been tossing around the idea of a NAS using a hardware based RAID, a software based RAID such as unRAID... or just getting something more turn-key like Synology.

I know people argue that a RAID is not a backup and I don't want to get into that discussion here; I agree and will maintain a separate backup. However, I'm just not sure that how I was doing things in the past (local storage on the STV server) is the way to go. Thoughts? Recommendations?

So, based on your collective experiences, I'm looking for opinions on... How would you do it today if you were starting over? Would you do something different than you have now?

EDIT: One thing, possibly worth consideration, is that I'll likely have up to 6 clients. I know that has been 1 reservation of mine with unRAID... not knowing weather or not the software based setup would be able to keep up, data wise, with the demand of that many clients - particularly if I were writing to PVR folders/drives as well.

[stuckless]

I've run many different raid solutions over the years, and unRAID is by far the best and easiest to setup and maintain. So, it gets my vote.

I don't that you need to worry to much about 6 clients... not sure how often you'll have all 6 clients running at the same time, but even still... i don't see that as being an issue.

If you did find that you are getting drive IO issues, you can setup unRAID shares to ONLY use certain drives, etc... I don't do that myself, since, I like the fact that the shares will use all drive space instead of being constrained by physical drive limitations.

The use of a Cache drive can help as well. I record to the cache drive myself, but you can choose what you want to put there.

Bottom line is that unRAID is user-friendly NAS solution with lots of options.

[Telecore]

I like keeping the media servers small, so I have been using Seagate STDR4000100 2.5 4TB HDD's (removing them from USB3.0 enclosure). I have two of these in my Win10 SageTV server for (1) recording and (2) MKVs. For mass storage, I use Stablebit Drivepool with an SSD cache in a separate file server (WHS 2011). I am trying Windows 10 Storage Spaces with our SageTV server at work, but so far I like Drivepool better. I use SecondCopy to move old recordings older than 120 days or so off the SageTV server and onto the file server. I currently have 6 HD-300's on-line but usually only 3 are on at a time worst case. No performance issues.

[Fuzzy]

I'd second stuckless's words on unRAID - and honestly, not because it's the best NAS solution out there - but because it's the best headless media server out there - that just happens to be built on top of a very functional and flexible NAS. Just about every major media server type software I've come across (except PlayOn, which explicitly depends on hacking into windows graphics system) has an easy to configure template that can very easily be just dropped onto it, and it 'just works'.

[drvnbysound]

Originally Posted by stuckless

I've run many different raid solutions over the years, and unRAID is by far the best and easiest to setup and maintain. So, it gets my vote.

I don't that you need to worry to much about 6 clients... not sure how often you'll have all 6 clients running at the same time, but even still... i don't see that as being an issue.

If you did find that you are getting drive IO issues, you can setup unRAID shares to ONLY use certain drives, etc... I don't do that myself, since, I like the fact that the shares will use all drive space instead of being constrained by physical drive limitations.

The use of a Cache drive can help as well. I record to the cache drive myself, but you can choose what you want to put there.

Bottom line is that unRAID is user-friendly NAS solution with lots of options.

Yeah, fairly unlikely that I would have all 6 running simultaneously but it would be "possible"... so I just want to ensure that I don't limit myself in terms of capability there. I had looked into using unRAID some time ago, and actually bought a (3) 5.25" to (5) 3.5" bay adapter for the mid-tower case I have as well. I just never pulled the trigger to purchase a license; just didn't have the time to do it either.

Thanks for the assurance!

One thing I do recall with unRAID is the number of different setup options (e.g. allocation methods, folder structure of shares). I know I'd need to dig back into that and get familiar with those things. Having said that, do you have any particular setup options to recommend? I probably need to search for a setup guide for multimedia applications.

[wayner]

I have 8-9 extenders in my house but only use a few simultaneously. I am looking to move from a Win server to a Linux server so that I can run 64 bit JVM and use a lot more heap space that should end SageTV crashing due to running our of JVM heap space.

I am likely moving to unRAID as well since it runs the 64 bit version of Java.

When you talk about DVD rips do you transcode to a different format like mkv or mp4 or do you keep the original DVD file structure? If it is the latter you may want to think about changing that as the Android TV client doesn't support playback of DVD files and that is likely the future for clients. Plus there is the fact that MPEG-2 isn't as well supported on modern devices.

[drvnbysound]

Originally Posted by wayner

I have 8-9 extenders in my house but only use a few simultaneously. I am looking to move from a Win server to a Linux server so that I can run 64 bit JVM and use a lot more heap space that should end SageTV crashing due to running our of JVM heap space.

I am likely moving to unRAID as well since it runs the 64 bit version of Java.

When you talk about DVD rips do you transcode to a different format like mkv or mp4 or do you keep the original DVD file structure? If it is the latter you may want to think about changing that as the Android TV client doesn't support playback of DVD files and that is likely the future for clients. Plus there is the fact that MPEG-2 isn't as well supported on modern devices.

UGH! That sucks! Yeah, I've done standard rips using the original DVD file structure (VIDEO_TS folders). I'll certainly need to play around with that a bit, but honestly, aside for the storage I need to re-consider my front end. I've previously been using HD200 extenders, but they are obviously out dated at this point. I need to play with the mini-client some, and have also been considering using a Kodi front-end if I can get that to point to (and obviously play) the DVD rips.

One thing to add to the wrinkle is that I do want to maintain is the ability to have rips that have 5.1 (or better) audio tracks [living room will have 5.1] as well as being able to have standard stereo [2.0] for the cases of using built-in TV speakers as well.

Apparently a lot to be done...

[wayner]

I use primarily HD-200s as well and so far there is little reason to change them since they work so well. I also have two Hd-300s. The main downside compared to Android TV is that they don't offer other stuff like Netflix.

I could be wrong about the DVD rips but I know it doesn't play DVDs. I took that to mean DVD VIDEO_TS structures but it might just mean the physical disk. Stuckless should respond if I am wrong as he is the god of this stuff.

[Fuzzy]

Originally Posted by wayner

I use primarily HD-200s as well and so far there is little reason to change them since they work so well. I also have two Hd-300s. The main downside compared to Android TV is that they don't offer other stuff like Netflix.

I could be wrong about the DVD rips but I know it doesn't play DVDs. I took that to mean DVD VIDEO_TS structures but it might just mean the physical disk. Stuckless should respond if I am wrong as he is the god of this stuff.

VIDEO_TS don't play, because the DVD Menu functions are not supported. Something like MakeMKV does a good and fast job of pulling the main movie out of the VIDEO_TS folders, and I believe I've seen scripting around to do so. Won't help with the MPEG-2 format, as MakeMKV just remuxes it, but it is quick. Handbrake, I believe, can also be scripted to pull the main movie, and it would be able to do a really good h.264 conversion as well.

[Fuzzy]

Originally Posted by drvnbysound

One thing I do recall with unRAID is the number of different setup options (e.g. allocation methods, folder structure of shares). I know I'd need to dig back into that and get familiar with those things. Having said that, do you have any particular setup options to recommend? I probably need to search for a setup guide for multimedia applications.

Allocation methods don't really matter much when dealing with SageTV, simply because sage aggregates everything into a singular database anyway, so what 'drive' it's on in the NAS is of no real consequence. I have a single import folder, and a single recording folder, both are configured to 'use cache'. This means writes of new files will go to the cache drive, and then be moved to the pool later (when the 'mover' runs). This does a nice job of allowing lots of different shows to be watched (read from the pool, which is very fast), while not impacting the writing of current recordings, that are written to the much faster SSD cache drive.

[NetworkGuy]

One point to consider, is to always record to locally attached storage.

I have tried recording to a NAS using a gigabit switch and still had problems. The NAS is great for playback, but not recording.

[wayner]

By the way for batch ripping of DVDs I found this script online. It might require a bit of tuning up for the latest version of Handbrake but it could be quite useful.

Batch Conversion of DVDs with Handbrake
January 14, 2015
I recently had a project to backup over 150 DVD titles to network storage. To save space, I decided to convert them all from DVD to h264. This cut the file sizes down from 4GB to about 1GB each. Processing each took around 20-30 minutes on my i7-4790k so I wanted to automate the process.

Below is the script I wrote to do everything for me. To use this yourself, just install Handbrake for Windows and set the user editable options seen at the top of the script and run it!

[SomeWhatLost]

Originally Posted by NetworkGuy

One point to consider, is to always record to locally attached storage.

I have tried recording to a NAS using a gigabit switch and still had problems. The NAS is great for playback, but not recording.

that is the joy of unRAID plus the sageTV docker... you are technically doing both

[wayner]

But that is no different from WHS, or the way that many folks have their systems set up where their SageTV server is their de facto NAS.

[drvnbysound]

Originally Posted by Fuzzy

VIDEO_TS don't play, because the DVD Menu functions are not supported. Something like MakeMKV does a good and fast job of pulling the main movie out of the VIDEO_TS folders, and I believe I've seen scripting around to do so. Won't help with the MPEG-2 format, as MakeMKV just remuxes it, but it is quick. Handbrake, I believe, can also be scripted to pull the main movie, and it would be able to do a really good h.264 conversion as well.

Hmm, nothing that can't be overcome, but sucks that I'm 99% sure that I've removed all DVD menu's as part of my ripping process, but they are still in the VIDEO_TS folder structure and I'm certain that they are all in MPEG-2 format.

[SomeWhatLost]

Originally Posted by wayner

But that is no different from WHS, or the way that many folks have their systems set up where their SageTV server is their de facto NAS.

true...
but I am so done with windows...
I will keep win7 on my desktop for the foreseeable future, but I already have that setup to dual boot to kubuntu... one more wrong move on M$ part trying to force Win10 on me and I will be single booting kubuntu


plus unraid's data pool plus cache pool is so much cooler... and the webui is so much nicer...

[brianrent]

I use windows STORAGE SPACES and it works well for me. The reason I started using this is because on 2 separate occasions a recording drive went bad. This was a pain to get those shows back not knowing exactly what was lost as I had at the time 4 recording drives. I now have 3 STORAGE SPACES drives that are set up as a 2 way mirror. Since using this I had another hard drive go bad and I popped another in and it synced up automatically copying the contents from its mirror after I told it to use the new drive for this.

[Fuzzy]

Originally Posted by drvnbysound

Hmm, nothing that can't be overcome, but sucks that I'm 99% sure that I've removed all DVD menu's as part of my ripping process, but they are still in the VIDEO_TS folder structure and I'm certain that they are all in MPEG-2 format.

Not sure WHY you would keep the complicated VIDEO_TS format for just a single movie file with no menus.

[drvnbysound]

Originally Posted by Fuzzy

Not sure WHY you would keep the complicated VIDEO_TS format for just a single movie file with no menus.

Stupidly simple ripping process with CloneDVD while also maintaining all audio formats...

Step 1:
http://cocoontech.com/forums/index.p...=1064&file=max

Step 2 (Default, unchecked option for preserving menus):
http://cocoontech.com/forums/index.p...=1065&file=max

Step 3 (Preserve 2.0 and 5.1 audio tracks):
http://cocoontech.com/forums/index.p...=1063&file=max

That's just been how I've been doing it for the past ~8 years or so. It's been an "easy button" for me.

[Steve20A]

Originally Posted by Fuzzy

I have a single import folder, and a single recording folder, both are configured to 'use cache'.

Fuzzy,

I like that idea. But, can you point me to docs that describe how you "configure to use cache?" I don't see an obvious reference in the Sage V7 docs. Also, some document direction for the "mover" function.

Or is this all your own code?

Thanks.

Steve