SageTV and Network Shares

[ptaylor]

I've been running low on drive space recently, so I decided to try something that I thought someone in these forums suggested.

I installed NASLite2 on a spare machine (which I'll refer to as NAS from now on) with two hard drives. I then moved all the content previously recorded by Sage to shares on the NAS. I then changed SageTV so that the video recording directories are on these NAS shares. Finally, I removed a 200 GB drive (that previously held content) from the SageTV server and installed it in the NAS also. The NAS now has a total of about 700 GB of space, probably 300 GB full.

I tested at various points throughout, ensuring that I didn't have any issues watching a Live HD stream that was written across the network on the NAS. Everything seemed fine. Note that I stated that I tested watching *A* Live HD stream... That became very important later.

I finished everything up about 8:30 this evening and the first real recording to take place at 9. What do you know? There were two HD programs scheduled to record at 9, in fact... (Yes, my brain heard a distance siren once I realized that two HD shows were scheduled at the same time...)

Well, I've had a chance to look through these two shows now and boy is it ugly. I'm guessing that the two HD streams are having problems recording simultaneously across the network. I don't know if the network card on the Sage machine can't keep up, or if the NAS can't write fast enough, or just what. There was serious stutter throughout the first 45 minutes of the first show that I viewed. After that, it mysteriously cleared up and looked fine for the last 15 minutes. The second show was not as bad as the first one, but it wasn't pretty either. I didn't try to watch this entire show to see if it mysteriously cleaned up completely along the way somewhere.

So, is the issue that two HD streams can't be recorded to a network share at the same time? Anyone else tried this in the past, or are doing this now?

For the time being, I've modified Sage's configuration so that it will not try to write to the empty NAS share and added a drive local to the SageTV server that is practically empty back as a recordable directory. (This empty drive has about 150 GB free, more free space than either of the two writable drives in the NAS, so Sage should prefer it to record on until another drive catches up, if my guess is correct.)

Any help would be greatly appreciated.

Thanks,
Paul

[mickp]

Try copying a few large files from your pvr to the nas and monitor your cpu while this is happening. It's also interesting to go to the view menu in task manager and turn on the option for "show kernel times". If when you copy files to the network there's a lot of cpu activity (and if this doesn't happen so much when copying files locally) then it might help to update your drivers.

Another thing worth checking if you run antivirus is that you don't have it set to scan files on the network, or at least have an exception for the nas' dir.

If that doesn't help then you probably want to run some benchmarks on the nas device. I haven't done much of this lately but I'm sure therere are plenty of decent apps out there which will give you an idea if there's a problem. Testing this from multiple clients may also help to indicate where the problem lies.



Mick.

[ptaylor]

I hadn't thought of the CPU usage due to the NIC... I haven't done anything to watch the CPU on the Sage machine when recording in a long time since it's all hardware. The NIC built into this box is a 3com 3c940 I believe. It's possible that the NIC drivers aren't very efficient and are causing the issue.

I'm not running A/V on this machine - It sucks away way too much CPU for something that is so time sensitive... (At least, McAfee does)

Another thought that I had was the NIC in the NAS itself. It's currently using a Realtek NIC and I have a spare Intel. (Just not sure if it is supported by the NAS software - I used the Realtek because I had read that it was supported)...

It is also possible that the NAS is CPU bound with it trying to write two HD streams to the same drive at the same time... It only has a 733 Mhz P3 processor. I'll have to try more testing tonight and see if I can narrow it down some.

[FidgetyRat]

I don't see why the CPU would be a huge bottleneck on the NAS unless you are running software RAID. Software Raid can use quite a bit of the systems hardware resources.

I would definitely try to pound the NAS and check its CPU usage.

[lovingHDTV]

I would guess that it is write access speed. I can stream from my unRaid two HD streams on my 100Mb network without any issues. I can however only write a single stream.

I've not found anyone that can. Most NAS storage users are not trying HD, but SD only. I have no problems at all with SD.

I ended up leaving a drive in my Sage system for HD storage. I know one user that does this, then uses DirMon to move the show after it is done recording to the NAS box.

Hope this helps,

[jkohn]

I think drive fragmentation can also play a part; when multiple streams are recording and the disk is getting thrashed due to fragmentation, you're more likely to have problems.

I have a total of 4 tuners. Recording two HD programs is no problem for me, and usually 3 is OK although there can be some issues sometimes. On Thurs there are 4 programs I need to record at once and this is far more problematic. One or two of the shows will be OK, but at least one will have some breakups in it. On a hunch I ran the fragmentation analysis in Windows Defrag on my two recording drives, and it shows significant fragmentation (which is surprising considering the drives in question are only about 1/3 full). So I've set up scheduled task to defrag the drives in the morning and I'll see if this makes a difference.

One other thing, SageTV isn't smart enough to load-balance recordings, it always records to the drive with the most free space. So when you initially copied all your recordings over to the NAS if they weren't evenly distributed over the drives it may be that Sage is recording multiple streams to a single drive which can increase the chances of problems.

[BobPhoenix]

Originally Posted by jkohn

One other thing, SageTV isn't smart enough to load-balance recordings, it always records to the drive with the most free space. So when you initially copied all your recordings over to the NAS if they weren't evenly distributed over the drives it may be that Sage is recording multiple streams to a single drive which can increase the chances of problems.

Actually you can force SageTV to use specific recording directories for specific tuners with the properties file: "forced_video_storage_path_prefix" - see this thread for a discussion.

[jkohn]

Originally Posted by BobPhoenix

Actually you can force SageTV to use specific recording directories for specific tuners with the properties file: "forced_video_storage_path_prefix" - see this thread for a discussion.

The problem with that is that all the recordings from those times when you're only recording one show at a time will end up on the same drive, which is not optimal.

[tmiranda]

ptaylor,

I use NASLite as well. I know for a fact that when you boot the NAS box it will automatically do all the SMART tests on all the drives even if you have SMART disabled. (Tony over at ServerElements told me that.)

When the SMART tests are being run you will get stuttering even when recording two SD streams (well, at least I do.) Once the SMART tests are done things should clear up.

Make sure the NASLite box is up for a while before running any tests. My box has 4 drives and the SMART testing takes about 30-45 minutes. (You mentioned in your fist post that things clear up after 45 minutes...) I keep SMART disabled to make sure there are no problems. (Remember to SAVE the config after making any changes.)

Regards,

Tom

[Lucas]

Originally Posted by ptaylor

I hadn't thought of the CPU usage due to the NIC... I haven't done anything to watch the CPU on the Sage machine when recording in a long time since it's all hardware. The NIC built into this box is a 3com 3c940 I believe. It's possible that the NIC drivers aren't very efficient and are causing the issue.

I'm not running A/V on this machine - It sucks away way too much CPU for something that is so time sensitive... (At least, McAfee does)

Another thought that I had was the NIC in the NAS itself. It's currently using a Realtek NIC and I have a spare Intel. (Just not sure if it is supported by the NAS software - I used the Realtek because I had read that it was supported)...

It is also possible that the NAS is CPU bound with it trying to write two HD streams to the same drive at the same time... It only has a 733 Mhz P3 processor. I'll have to try more testing tonight and see if I can narrow it down some.

733Mhz is plenty for NASlite.

I think your problems are disk(NAS) throughput related at the time of the recording.

Since, as you say, one HD stream is no problem then you can do the following.

Point each of you HDTV capture cards to a specific Naslite disk(there is a relevant Sage.properties entry) and make sure that each disk is on a separate IDE channel.

Each concurrent show will be recorded on a separate disk. If this doesn't work then you have problems due to the network being the bottleneck.

[ptaylor]

Just an update...

What I'm doing at this point:

One of the reasons for this NAS move in the first place was to let me easily rebuild the OS of the Sage machine. I envisioned having a clean install of SageTV with only the things I actually need for Sage installed on this machine. (I've "played" with several programs on this install over the last year and a half or so.)

I'm using a 40 GB drive just for the OS. I happen to have a spare 40 GB drive and I've installed it just below the primary boot drive. So, when I have a few hours with no recordings scheduled, I'm shutting the box down, moving the cable to the other 40 GB drive and booting back up. So far, I've installed the base OS this way along with VNC, the audio drivers, and the latest NIC drivers (as it turned out, there were newer ones that I was running, so that could be related to my issue also). I still have tons of stuff to do (Windows Activation, disable MCE, install drivers for the video card and video capture cards, install Sage, configure some Windows settings, etc.).

Last night I was coming up on 8 PM, so I shut the machine back down, moved the IDE and power cable over and booted back up.

Once I get everything installed and configured the way I want, I'll just stop using the new drive and make an image of the installation. If I lose the drive, just pop in another drive, restore the image and I'm back to "normal". I'll need to take new images when I update to newer versions of Sage, but the idea is to make it an appliance.

Gotta run...

[BBCritical]

Does anyone have success with livetv and naslite without any stuttering? Im having this problem on a 1 Gb network...

[ptaylor]

I think I've seen posts indicating that there weren't issues with two streams of SD content. Personally, when I tried a single stream at a time, even with HD content, it looked fine... But two HD streams just completely stunk...

I've actually reverted to simply using my old Rebyte for my NAS needs (as I have three 40 GB drives running in RAID5 with it for my critical storage needs).

As for my video storage needs, I'm just going to cram as many drives into my main Sage machine as it will handle, as I need more room... (The motherboard will take the standard 4 IDE connections, so that's a DVD drive and three hard drives, plus it has two SATA controllers built in.)