Building new unRAID server - any advice?

[wayner]

My unRAID server is now about 5 years old and it is starting to have some issues - I just had to replace the parity drive. This system is running unRAID 6.4.1 and it uses the old DVB edition plugin to have drivers for HD-PVRs - this has now been deprecated. I am worried that other components, like the SSD used as the cache drive, or the system USB thumbdrive might fail.

I want to build a new system with new hardware. Anyone have any advice on building a system that will be used for SageTV and as a file server? My tuners will be 2 or 3 HD-PVRs, and a HDHR OTA tuner. The system will run a bunch of dockers in addition to SageTV. I will need the DVB edition in order to use the HD-PVRs. I will run an Ubuntu VM and I might want to run a Windows 10/11 VM as well. I will keep the old hardware around as a backup server

Here is what I am thinking
Rackmount server case (I have a rack with my Home automation equipment)
Mobo Asus B560M-A
CPU Intel i5-10500
Memory - 2x16GB for a total of 32GB
Hard drives - 3x8TB drives (one as parity)
SSD 500GB NVME SSD as Cache drive

Anyone have an opinion on how large the cache drive should be? My old server has a 240GB SSD. But there doesn't seem much point in going below 500GB. Should I go 1TB? Do the NVME drives make a difference?

Is 32GB the right amount of memory? Should I go higher? Or is 16TB enough?

Anything else to consider.

For tuning my cable boxes I will either use Global Cache devices or a might use firewire.

Here is the hardware config of my old system:

System Overview	
Unraid system:	Unraid server Plus, version 6.4.1
Model:	Custom
Motherboard:	ASUSTeK Computer INC. - P8H67-M EVO
Processor:	Intel® Core™ i5-2500K CPU @ 3.30GHz
HVM:	Enabled
IOMMU:	Disabled
Cache:	L1-Cache = 256 kB (max. capacity 256 kB)
	L2-Cache = 1024 kB (max. capacity 1024 kB)
	L3-Cache = 6144 kB (max. capacity 6144 kB)
Memory:	24 GB (max. installable capacity 32 GB)
	ChannelA-DIMM0 = 8192 MB, 1333 MHz
	ChannelA-DIMM1 = 4096 MB, 1333 MHz
	ChannelB-DIMM0 = 8192 MB, 1333 MHz
	ChannelB-DIMM1 = 4096 MB, 1333 MHz
Network:	bond0: fault-tolerance (active-backup), mtu 1500
	eth0: 1000Mb/s, full duplex, mtu 1500
Kernel:	Linux 4.14.16-unRAID x86_64
OpenSSL:	1.0.2n
P + Q algorithm:	9589 MB/s + 13886 MB/s
Uptime:	2 days, 5 hours, 53 minutes, 37 seconds

[stuckless]

My only advice would be to add a second 512g SSD into the cache pool (for parity). That will give you fail protection if one of them goes. You are already going with 8tb parity drives, so that is good (I made the mistake of using 3tb parity drives, but I'm currently moving to 8).

Btw, even your "old" system is quite decent for unRAID

I'm currently running

AMD FX™-8350 Eight-Core @ 4000 MHz (2012)
24gb memory
12tb storage (soon to be 18)

I run 18 docker containers - 11 on full time, and 7 are just spin up when I need them.
I run 1 windows VM - I rarely start it up, but from time to time I need to use it for some stuff.

[wayner]

Thanks Stuckless. Good idea on the second Cache drive. I think that this mobo has two M.2 SATA slots so I can throw 2x500GB for Cache. Or is it a waste to use 512GB - is 256GB enough? It looks like 5112GB M.2 SATAs start around

My current system is OK, but I worry that I am going to start to have more components failing. I also want to move the system into my server rack and I don't want to deal with having to physically rebuild the existing system (mobo, drives, etc) into a new case. I also like having a "hot" backup and my backup was my previous Windows SageTV server that is now over a decade old.

One thing that I am never sure about is how much space/memory you allocate for Docker. I occasionally get messages that the Docker volume is over 80% full. What exactly goes into the Docker volume? Is this all of the system folders for SageTV (and other Dockers)? In the case of SageTV this tends to grow over time as your Fanart and thumbnails can become many GB in size.

[egeller]

Originally Posted by wayner

I occasionally get messages that the Docker volume is over 80% full. What exactly goes into the Docker volume? Is this all of the system folders for SageTV (and other Dockers)? In the case of SageTV this tends to grow over time as your Fanart and thumbnails can become many GB in size.

Docker volume is not Fanart. Docker volume is a file containing all the Docker images. It should be small, but can get big. I had the same problem a year or two ago.

I had messages about Docker size being over 80%. It took some time but I determined the culprit was OpenDCT. I Telnet'ed into the OpenDCT docker, while it is running and checked the /tmp folder. It contained a lot of subfolders whose names started with "javacpp". I think they're temp directories. I deleted them (all but the newest one) and freed up quite a bit of space in the Docker image. Sometimes OpenDCT cleans up after itself, but, in my experience, it usually does not. Now I check it periodically.

[KeithAbbott]

Originally Posted by wayner

Anyone have an opinion on how large the cache drive should be? My old server has a 240GB SSD. But there doesn't seem much point in going below 500GB. Should I go 1TB? Do the NVME drives make a difference?

I have dual 500GB NVME cache drives, if I were to do it over again I would probably save some money and use 240GB drives. I have the mover set to run daily overnight. If you find you are filling up the cache drive, you could always set it to run hourly. The only time I had to do that was during the Olympics, but I did it more to keep the NVME drive temperatures down, as I was getting warnings when it was moving many many gigabytes of data only once per day.

You might also consider getting a SAS controller, I have a LSI 9207-8i SAS Host Bus Adapter. With the right case and cables, it can really help with cable management inside the server case. I have six spinning drives, cable management would be insane if I was dealing with individual SATA cables.

[stuckless]

I would think that 240g ssd is more than enough for cache drive(s). Although, I might do 512g ones today, if I were starting over. Mainly because my windows vms on the cache drive simply for performance. (my docker.img is also on cache). The share holds my docker/vms is set to cache-only. That being said, I have 192g of cache and it's 80% used.

As for Docker, when I setup unraid I set my docker.img to be 50g. Right now, I'm using ~20g of the 50g.

devid    1 size 50.00GiB used 19.52GiB path /dev/loop2

I also have my appdata share set to prefer cache and my sagerecordings was set to cache (meaning that I'd record to cache and then they get moved to the pool later)

My appdata is on prefer cache so that my plex library and other configs (sagetv, etc) all run from cache only for performance. So, if you are setting you appdata to cache then I can see it filling up the cache with sagetv fanart, etc. But docker itself will never take more than the initial value you give it (50g).

Because of the amount of critical data I have in cache only, I use 2 ssds to prevent data loss in the event of failure.

[wayner]

Thanks for the info - it was five years ago that I built my unRAID server and I haven't focused on this so you tend to forget why you set it up in a certain way. My appdata is also set to prefer cache - I imagine that should speed up the SageTV clients.

I don't have my recordings using the cache. I only have one media share set up for all types of media - TV recordings, movie files, photos and music.

How do you use the cache for recordings? Do you set up an additional share called TV Recordings and set this to Prefer Share?

[wayner]

One more question on use the cache drive for recordings. Do you use the "Yes" option for cache, rather than Prefer which is used for appdata? And if you aren't have any issues today, is there any reason to use the Cache drive for recordings. I rarely record more than two simultaneous HD-PVR streams and I don't think I have ever had issues when it comes to writing TV recording files to disk.

Yes indicates that all new files and subdirectories should be written to the Cache disk/pool, provided enough free space exists on the Cache disk/pool. If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array. When the mover is invoked, files and subdirectories are transferred off the Cache disk/pool and onto the array.

[stuckless]

For my sagemedia share (where recordings happen) it is set to cache=yes, meaning that it's temporary cache and files eventually get moved.

In the unRAID UI an often missed feature is that if you mouse over the labels in the setting, there is help if you click on the label (which it is showing the help icon).

For the "Use Cache Pool" option, they explain the 3 options (much better than I can).

Specify whether new files and directories written on the share can be written onto the Cache disk/pool if present. This setting also affects mover behavior.

No prohibits new files and subdirectories from being written onto the Cache disk/pool. Mover will take no action so any existing files for this share that are on the cache are left there.

Yes indicates that all new files and subdirectories should be written to the Cache disk/pool, provided enough free space exists on the Cache disk/pool. If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array. When the mover is invoked, files and subdirectories are transferred off the Cache disk/pool and onto the array.

Only indicates that all new files and subdirectories must be writen to the Cache disk/pool. If there is insufficient free space on the Cache disk/pool, create operations will fail with out of space status. Mover will take no action so any existing files for this share that are on the array are left there.

Prefer indicates that all new files and subdirectories should be written to the Cache disk/pool, provided enough free space exists on the Cache disk/pool. If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array. When the mover is invoked, files and subdirectories are transferred off the array and onto the Cache disk/pool.

NOTE: Mover will never move any files that are currently in use. This means if you want to move files associated with system services such as Docker or VMs then you need to disable these services while mover is running.

IF I had a larger cache drive I would set my appdata to cache-only but since I have small one (cache drive) I set it to prefer, meaning that as cache becomes available files get moved from the pool to cache, but if there isn't enough room on cache, then they end up in the pool. So cache only and cache prefer is very similar just that cache only can run out room.

As for WHY I set to my recordings to prefer, was really to just to minimize disk IO. ie, I almost never watch a recording as it is being recorded, so normally I'm writing 2 shows to cache, and then watching another show, so that show that is being watched is coming from the pool, meaning that my IO for watching and recording is distributed. Whether or not it makes a difference... not sure.

[wayner]

Thanks for the info. Does Mover run automatically every X hours, or overnight? I seem to recall that this has changed over the years in how unRAID works.

[egeller]

It's configurable right from the Main tab. I think it was every 24 hours, when I set up my servers. I run it every hour. I don't want recordings to stay on the cache longer than that.

[wayner]

Here's a good video by Space Invader that has a lot of info on configuring drives, cache, etc on unRAID 6.9 https://www.youtube.com/watch?v=jgRSr7yBZfs

[wayner]

Anyone know if using a Gen4 M.2 NVME SSD will provide benefits over a Gen3 NVME SSD for my cache drive? Or are these both so fast that a 1GB network is going to be the bottleneck rather than pulling data from an already fast SSD?

I do remember that when I first went to an SSD for my SageTV server years ago that it provided noticeable speed improvements.

These days what would be the most important things to get SageTV running as zippily as possible when I am using HD300s as clients? Will that change if I switch to ATV boxes as clients?