Appropriate hardware

[mlcarson]

Will this hardware work for a docker install? Should I even be considering it or just do a normal Linux install.

I3-3225
Intel DH 77DF mainboard
8GB RAM
250GB SSD
8TB WD80DFZX (2.61TB MP4 files currently used)
HDHomerun Prime x 2

Clients: Android Nvidia Shield.

I've been using Windows 10 64-bit for my SageTV installation until now but have the above hardware I can dedicate to it and was just considering a base Linux installation -- no unraid. Starting dockers would be SageTV and OpenDCT.

An alternative would be an i5-2500K, 16GB RAM solution but I wanted to keep that as a Windows solution where the above could be Linux. No Unraid in the setup. Always figured an actual backup was better than a RAID setup.

[Fuzzy]

Originally Posted by mlcarson

Will this hardware work for a docker install? Should I even be considering it or just do a normal Linux install.

I3-3225
Intel DH 77DF mainboard
8GB RAM
250GB SSD
8TB WD80DFZX (2.61TB MP4 files currently used)
HDHomerun Prime x 2

Clients: Android Nvidia Shield.

I've been using Windows 10 64-bit for my SageTV installation until now but have the above hardware I can dedicate to it and was just considering a base Linux installation -- no unraid. Starting dockers would be SageTV and OpenDCT.

An alternative would be an i5-2500K, 16GB RAM solution but I wanted to keep that as a Windows solution where the above could be Linux. No Unraid in the setup. Always figured an actual backup was better than a RAID setup.

That would 'work', but you certainly wouldn't benefit from the parity in unRAID with a single massive drive (unRAID is best suited to many smaller drives). Personally, I like the web UI of unRAID, which for me is much nicer for most general setup work than a bare linux install would be, but if you are comfortable enough in linux to install that way, it might not be worth the money for unRAID.

[EnterNoEscape]

Originally Posted by Fuzzy

That would 'work', but you certainly wouldn't benefit from the parity in unRAID with a single massive drive (unRAID is best suited to many smaller drives). Personally, I like the web UI of unRAID, which for me is much nicer for most general setup work than a bare linux install would be, but if you are comfortable enough in linux to install that way, it might not be worth the money for unRAID.

I'm very comfortable with the command line in Linux and I still strongly prefer the web interface of unRAID for docker. I haven't found anything that makes working with containers the way a normal home user would work with them as trivial as unRAID.

On my CentOS boxes, I have scripts to set up the containers with all of the needed parameters. CentOS offers a web interface for general system administration called Cockpit which is really nice, but it doesn't get down to the level of simplicity that unRAID gives you in regards to docker.

[mlcarson]

Sounds like the docker container then is just something to avoid and I can just do a normal Linux installation.

[EnterNoEscape]

Originally Posted by mlcarson

Sounds like the docker container then is just something to avoid and I can just do a normal Linux installation.

I would not go that far. Most docker repositories will actually give you a complete command line to get any container they provide working. You just need to be a little organized so that you don't need to go back and redo the paths for example.

[mlcarson]

Is Docker the intended way of installing SageTV now? The threads seem to indicate some issues yet.

I'm not a believer in Raid for the home (even Unraid). It's never worked out for me in the past. Hard drive pricing just continues to go down while capacities go up. You end up spending more for the same capacity and end up with a false sense of security regarding the safety of your data due to the redundancy if you don't do proper backups. If you do proper backups then there's really no need for the redundancy in the first place like there is in an enterprise environment.

Most Linux installations support Docker and there are some very tiny distributions. Why not direct people to something like CoreOS or Alpine Linux? Granted Unraid Basic at $59 isn't that expensive but it shouldn't be an expected part of a SageTV install.

I currently have my SageTV installation on Windows 10 Pro 64-bit but it's just been in test mode. I'm now looking at moving off from my Tivo system. The Tivo includes a Tivo Roamio Pro and 2 Mini's. The new SageTV setup has 3 Nvidia Shield's, 2 HDHomerun Prime tuners, and an i3-3225/8GB/8TB server. The server was a Plex server but I don't see the need for it with SageTV. It was a convenient place to put my video collection that the Tivo's could access but SageTV can do that natively so am using the hardware as a dedicated SageTV server.

I haven't noticed a JVM heap size issue yet on Windows but I haven't been using SageTV exclusively either. This would be the perfect time to move to Linux if it's going to become an issue but if it's not, it's a lot easier to just stick with Windows 10 since it's already installed.

So which way should I go? Keep Windows 10, install Linux (no docker), or install Linux and use the docker images? If Docker, I need both the SageTV and the OpenDCT because of the HDHomerun Primes.

[stanger89]

Originally Posted by mlcarson

Is Docker the intended way of installing SageTV now? The threads seem to indicate some issues yet.

I wouldn't say "intended", but definitely it's a popular one due to the ease of install. As for issues, I don't know, I finally switched over to unRAID/docker for my "production" SageTV system and it's running fine.

I would guess a lot of the issues are people working out the changeover. I think most SageTV users, like I, were using the Windows version because prior to going open source, the Linux version of Sage was "OEM only". There's a bit of a learning curve, to switching over, mostly from finding new ways to do things that aren't done the same on Linux.

I'm not a believer in Raid for the home (even Unraid). It's never worked out for me in the past. Hard drive pricing just continues to go down while capacities go up. You end up spending more for the same capacity and end up with a false sense of security regarding the safety of your data due to the redundancy if you don't do proper backups. If you do proper backups then there's really no need for the redundancy in the first place like there is in an enterprise environment.

First let me say that I agree completely about the importance of backups. I have Crashplan running, backing up my important, irreplaceable data. I'm a big fan of unRAID, so I'll make a couple of points:


1) "You end up spending more for the same capacity" - This isn't correct, not with unRAID. I just looked into adding capacity to my unRAID server, in the end, I ended up getting a 4TB drive to replace an older 2.5TB one. I looked at drives ranging from that all the way up to 10TB, and right now 4TB is about the sweet spot price wise, but 10TBs aren't far off ($/GB). The great thing about unRAID is you can always migrate to larger drives, and you don't have to migrate the whole array at the same time, all you need have to do is the parity(s) and one data drive. And you can upgrade at any time.

2) "If you do proper backups then there's really no need for the redundancy in the first place like there is in an enterprise environment." - I won't disagree on "need" but I do disagree on the value of redundancy. My unRAID box is (IIRC) about 40TB these days. A small portion of that is irreplaceable and backed up. The majority of it is hundreds of ripped movies and thousands of TV episodes. I've put a lot of time and effort into ripping and organizing them. The parity/redundancy IMO provides a valuable layer of protection against the most likely form of data loss, drive failure. Parity protection means I don't have to re-rip anything if I do lose a drive.

Of course the biggest benefit, my favorite part is the pooling. I have 12 data drives in my array, and with pooling, I don't have to worry about which drive what files are on, I don't have to worry about which drive is free when I'm ripping stuff, etc. It just makes for a much easier experience.

Most Linux installations support Docker and there are some very tiny distributions. Why not direct people to something like CoreOS or Alpine Linux? Granted Unraid Basic at $59 isn't that expensive but it shouldn't be an expected part of a SageTV install.

Nobody's directing anyone to anything here. There are just a number of SageTV users who run unRAID, and unRAID has a very nice docker management system, and some folks have combined the two.

There's nothing in the SageTV software, nothing in the SageTV install that requires unRAID, or even Docker. The SageTV Docker container uses the standard SageTV install scripts, and the unRAID template is just that, at template for unRAID that points to the SageTV Docker container.

[tmiranda]

I've been watching all of the UnRAID discussions and have stuck with a "standard" linux install. I'm just too lazy to learn yet another OS (which UnRAID essentially is). For me UnRAID just doesn't offer any advantage.

I'm not a fan of many small drives, I usually go for fewer, larger, drives. Anything that I can't afford to lose get's copied to a NAS box that has RAID1.

I think people should just use what works best for them. If UnRAID offers something you want, or need, then use it. If not, then don't.

[KarylFStein]

Originally Posted by tmiranda

I've been watching all of the UnRAID discussions and have stuck with a "standard" linux install. I'm just too lazy to learn yet another OS (which UnRAID essentially is). For me UnRAID just doesn't offer any advantage.

I'm not a fan of many small drives, I usually go for fewer, larger, drives. Anything that I can't afford to lose get's copied to a NAS box that has RAID1.

I think people should just use what works best for them. If UnRAID offers something you want, or need, then use it. If not, then don't.

I don't use unRAID for SageTV yet, but find the docker/VM/disk management very nice so have been using it for some other things.

RAID I use for fault-tolerance. I've lost enough HDDs over the years that it's nice to be able to lose one and not lose service. "Important" data I store off-site.

[mlcarson]

How about the just keep using Windows option with 3 Android mini clients? Is 32-bit Java too limiting for this?

[mlcarson]

You always spend more in a RAID setup - even with Unraid - because of the parity disk. That parity disk has to be the largest disk in the system. There's definitely advantages to having redundancy but it's not free. I've done RAID1 and RAID5 in the past at home. For me, it's been simpler and more beneficial to just do backups to external disks and organize things such that volumes larger than the size of one disk haven't been necessary. Being in a degraded state on a raid array or suffering through the rebuild process is MORE painful than a restoration of a backup.

The pricing of hard drives is pretty linear right now on cost per TB except on the low end. If I had 40TB of data and most of it was non-changing, I'd probably still just do backups of the static data to 10TB drives and then just have these drives stored some place offsite. The restoration times for even 10TB are scary. I'm expecting 1-2 days. I'm betting the rebuild times of a drive in an Unraid system are pretty horrific too. Is such a system even usable during a rebuild?

Proper backups to me mean having a copy of the data on another hard drive preferably offsite but definitely offline and is independent of the redundancy level of a system. I think most home users choose either redundancy or a proper backup/restore strategy. Redundancy is optional; the backup/restore strategy isn't if the data is actually important. Redundancy is easier/cheaper and too many people choose it as an alternative to a backup solution when it really isn't.

Given all of that, Unraid is pretty cool on the application server/virtualization front. I like the Docker idea -- not sure if it's right for me at home with SageTV but I can see using it at work.

Originally Posted by stanger89

1) "You end up spending more for the same capacity" - This isn't correct, not with unRAID. I just looked into adding capacity to my unRAID server, in the end, I ended up getting a 4TB drive to replace an older 2.5TB one. I looked at drives ranging from that all the way up to 10TB, and right now 4TB is about the sweet spot price wise, but 10TBs aren't far off ($/GB). The great thing about unRAID is you can always migrate to larger drives, and you don't have to migrate the whole array at the same time, all you need have to do is the parity(s) and one data drive. And you can upgrade at any time.

2) "If you do proper backups then there's really no need for the redundancy in the first place like there is in an enterprise environment." - I won't disagree on "need" but I do disagree on the value of redundancy. My unRAID box is (IIRC) about 40TB these days. A small portion of that is irreplaceable and backed up. The majority of it is hundreds of ripped movies and thousands of TV episodes. I've put a lot of time and effort into ripping and organizing them. The parity/redundancy IMO provides a valuable layer of protection against the most likely form of data loss, drive failure. Parity protection means I don't have to re-rip anything if I do lose a drive.

Of course the biggest benefit, my favorite part is the pooling. I have 12 data drives in my array, and with pooling, I don't have to worry about which drive what files are on, I don't have to worry about which drive is free when I'm ripping stuff, etc. It just makes for a much easier experience.

Nobody's directing anyone to anything here. There are just a number of SageTV users who run unRAID, and unRAID has a very nice docker management system, and some folks have combined the two.

There's nothing in the SageTV software, nothing in the SageTV install that requires unRAID, or even Docker. The SageTV Docker container uses the standard SageTV install scripts, and the unRAID template is just that, at template for unRAID that points to the SageTV Docker container.

[jusjoken]

Originally Posted by mlcarson

How about the just keep using Windows option with 3 Android mini clients? Is 32-bit Java too limiting for this?

Depends. What UI do you plan to use on the extenders? I find that 3 is ok if you use the default UI but if you start adding a bunch of plugins or use Gemstone then 3 can push the limits of the heap. YMMV.

k

[mlcarson]

I've been using the default UI. The only additional stuff that I've really been using is BMT. It gives me the episode data that I was getting from Plex.

[Skirge01]

You may want to look into FlexRAID's tRAID software. It uses parity, but all drives are accessible outside of the array. You can assign a drive letter to any individual drive and pull data off of it using any computer. When a drive fails, you only need to use the parity to restore the data which was on that one drive, as opposed to the entire array. You can also have as many parity drives as you want, all the way up to what would essentially be a mirrored array, in order to protect against multiple drive failures. However, if you only have a single parity drive and a second data drive fails while you're restoring the first failed data drive, you only lose the data on the second data drive. Plus, the array and the remaining data is still available throughout all of that.

This flexibility of tRAID is what sold me on it.

[Fuzzy]

Also bear in mind, that the parity drive is not required in unRAID. If you don't want to spend the money on an n+1 drive, just don't install one. It will still work with all the pool, docker and VM capabilities without having a parity drive - and one can always be added later too if you just want to spread out the expense.

[Fuzzy]

Originally Posted by mlcarson

I've been using the default UI. The only additional stuff that I've really been using is BMT. It gives me the episode data that I was getting from Plex.

If you aren't running a lot of fanart in your UI, than you can easily run 3-5 miniclients in a 1GB JVM.

[mlcarson]

I think I'll just keep it Windows for now until I see issues since that's definitely the easier route.

[stanger89]

Originally Posted by mlcarson

You always spend more in a RAID setup - even with Unraid - because of the parity disk. That parity disk has to be the largest disk in the system. There's definitely advantages to having redundancy but it's not free.

I thought you were saying it costs more because once you pick a size, you're stuck with that size drive (ala RAID-1/RAID-5/etc) and it may below the $/GB curve.

I've done RAID1 and RAID5 in the past at home. For me, it's been simpler and more beneficial to just do backups to external disks and organize things such that volumes larger than the size of one disk haven't been necessary. Being in a degraded state on a raid array or suffering through the rebuild process is MORE painful than a restoration of a backup.

If you only need a drive or two, or three, individual drives aren't bad.

The pricing of hard drives is pretty linear right now on cost per TB except on the low end. If I had 40TB of data and most of it was non-changing, I'd probably still just do backups of the static data to 10TB drives and then just have these drives stored some place offsite.

Storage costs are ~$40/TB these days. If I wanted to just have a copy of it, that duplication would cost me ~$1600 ($40/TB* 40TB). That makes the extra ~$140 for a 4TB parity drive seem pretty reasonable.

The restoration times for even 10TB are scary. I'm expecting 1-2 days. I'm betting the rebuild times of a drive in an Unraid system are pretty horrific too. Is such a system even usable during a rebuild?

Yes it is, and unRAID 6 even supports dual parity, so your array can even be fault tolerant during that process.

Proper backups to me mean having a copy of the data on another hard drive preferably offsite but definitely offline and is independent of the redundancy level of a system.

Agreed completely. All my important data is backed up offsite. The redundancy feature of unRAID is largely a bonus on top of it's other features. And it's well worth the cost of a parity drive to save me weeks of work re-ripping content in the event of a hard drive failure, and relative to the cost of purchasing an extra 44TB of storage.

[mlcarson]

You seem to be agreeing but then saying that you haven't paid for the duplication cost of 40TB of data and have only purchased a 4TB parity drive. If you've done both then bravo - most people don't do that.

The point of the rebuilding rant is that it can potentially bring down your raid to an unusable status for longer than the time than it would have taken to just restore a backup. That was true for RAID5 -- not sure how Unraid behaves in that scenario. I suspect it has a lot to do with the CPU power available to the Unraid server. If RAID6 double parity is used, the rebuild should take even longer than with RAID5. So if you have decent backups, the question is what does the additional RAID system give you? If performance degradation during a rebuild is negligible then I get it. Back when I was doing RAID5, it was anything but.

[Fuzzy]

I don't believe the unRAID array needs to be offline to rebuild, though I haven't dug that deep into it. Although, use while rebuilding will certainly slow down the rebuilding process.