Win10 Client Stops Working after 2hrs

[johnw885]

I have 2 Win10 clients, one on my desktop that works fine, and another on an NUC that stops working after a couple hours or so. They both connect to a Linux sever in Vmware. After a couple of hrs a recorded TV show takes 30+ seconds to start or each time you fast forward. Often is just times out. I've rebuilt everything few times and can't figure out what's happening. Try to avoid going back to MCE but the misses is getting mad. Any help would be appreciated. I've included my logs.

[waynedunham]

Just a WAG (Wild a@@ guess), but is there a difference in the Java Heap size on the NUC?
I forget if java heap warnings get put in the system messages area in the native code or if that is done by the java heap monitor plugin I have installed on all my extenders.

When I've run into Java heap problems I have seen symptoms like you are seeing.

One of my Servers does also exhibit a noticeable delay on starting playback of recordings. I will see anywhere from no delay up to 5-10 seconds (probably 5, but it seems longer). I have never been able to figure out why. It is a more powerful cpu than the other server with no delays, more ram, and the same maxxed Java Heap setting.

[johnw885]

I thought it maybe a Java issues but the heap size on both the server and client are set to 1024. The client says it's only using 272Mb. The Linux server I'm not sure how much is in use, couldn't figure that out yet, but I'm sure the server is probably not much more than that. I didn't update the version sagetv installed, just changed the heap.

I do use Gemstone for the Recorded TV, it's the closest I've found to MCE RecordedTVHD plugin, but seems less snappy that the default. I also use comskip with all those plugins and the donators version.

I checked the speed on everything from end to end and it should be able to handle 6+ streams at low latency, so way more than one. It also plays just fine in WMP over the samba share in my recordedtv folder.

[stuckless]

Typically when I've seen long pauses in the UI... they boil down IO issues (disk usually), network issues (faulty ports), or java (process starving).

When the NUC experiences these issues, does the desktop also experience the same issues?

In the case of the Nuc, does it happen when you are watching something for a couple hours, and then go back to sagetv UI? Or just simply it's turned on, unused for a couple hours, and then you start using it?

I'm wondering if you have drives that the going into sleep/power save mode (spinning down). I experiences some issues like this myself, and it was a result of the drives being spun down, and then when I'd start to access things again, the sagetv would start to read files from the drive, and it would take 10+ seconds to spin up.

I don't really see anything in the logs to suggest a memory issue.

[waynedunham]

Originally Posted by johnw885

I thought it maybe a Java issues but the heap size on both the server and client are set to 1024. The client says it's only using 272Mb. The Linux server I'm not sure how much is in use, couldn't figure that out yet, but I'm sure the server is probably not much more than that. I didn't update the version sagetv installed, just changed the heap.

I do use Gemstone for the Recorded TV, it's the closest I've found to MCE RecordedTVHD plugin, but seems less snappy that the default. I also use comskip with all those plugins and the donators version.

I checked the speed on everything from end to end and it should be able to handle 6+ streams at low latency, so way more than one. It also plays just fine in WMP over the samba share in my recordedtv folder.

I'm not that familiar with the Linux SageTV experience, but on the Windows version you can see the Java heap info on the setup/system info screen down a couple pages. Right under the list of what is being recorded by all your tuners. Shows the current/max/limit. At least on the stock UI anyway, not sure about Gemstone.

I'm not sure how many tuners you have, but I had to limit the # of comskips that can run at the same time to 3 on my main system. Any more and I'd see slowdowns at times in responsiveness. I have a core2 quad running on WHS V1. I just set the limit the same on my i5 server to be safe.
So that might be another possible area that might cause slowdowns.

If your slowdowns are while watching something then disk spindown shouldn't be an issue, but it might be worth a look to check all your power settings on the windows machine.

You say it plays fine in WMP, is that at the same time you're seeing slowdowns in SageTV? If so then maybe check on what decoder you're using. I don't use a PC for playback, all my stuff is done via SageTV Extenders so that hasn't been an issue or me, but if you're playing back on a PC it can be.

[johnw885]

I thought maybe disk or network too but I tested the disk and it's getting around 80 MB/s. The network can push around 40MB/s to that disk with a latency of 2ms - 15ms so I'm fairly sure it's not either. These streams are around less than 2 MB/s each so I don't think it's an issue. I also never have issues playing a TS file from the samba share, it always works fine even when the sagetv client isn't.

The disks never go into power-save made since there in a Proliant server with that turned off. The issue starts happening and then doesn't stop for many hours, sometimes a client reload will fix it but that maybe coincidence since sometimes it doesn't.

The desktop never experiences any issues but most of my viewing is on the NUC to my TV. I'll go check it when the NUC is having issues but the desktop will be working fine.

I was learning towards codecs or drivers but it happens a few hours after a fresh install with everything updated, using LAV with bit streaming to a audio receiver.

The only other thing I can maybe think of is the comskip plugin stalling things.....? maybe, not sure how'd I diagnose that except to stop using it.

I'm having a hard time figuring what changes when it starts happening. Seems like nothing has changed but I'm not very skilled in the software side of things, I'm a vm and network guy, and I'm at loss on this one. I was hoping the logs would have something glaring.

[stuckless]

When you experience it again... connect to the linux server... (you mentioned that it's a linux vm, so is this a Linux VM running on Windows?) and run the "top" command. Top will show what processes are running, and amount of cpu + memory they are consuming... this might help narrow it down if your think it's the server.

[waynedunham]

Originally Posted by johnw885

8< snip >8
The only other thing I can maybe think of is the comskip plugin stalling things.....? maybe, not sure how'd I diagnose that except to stop using it.
.

It sounds like you have done a lot of diagnostic work. When the issue happens again, on both the NUC, and the server fire up task manager and hit the CPU tab to see if something is grabbing a lot of CPU.
What are you using to fire off Comskip? I use commecial detector. You can see a lot of info on what it is doing by going to detailed setup/customize. Then down near the bottom of that menu you'll see "Comskip Playback Options". Click on Modify. A couple up from the bottom of the menu you get will be "Commercial Detector". Click on "Configure". There you'll see all the options for commercial detector and how it runs comskip. You want the advanced options so you're sure to see all the options.
I also have the "run more slowly" (i.e. comskips Playnice option) selected on mine and a max of 3 jobs running. The run more slowly/playnice will cut down comskips CPU usage quite a bit. At least on my system it is still fast enough running 3 jobs to at least keep up to live if you're watching as you record.

A couple pages down you'll see the # of shows in the queue waiting to be comskipped, the line below that will clear the queue if you click on it. Just below that is the line with the # of jobs currently running. And just like the queue the line below that if you click on it that will kill all the running comskip jobs.

Commecial Detector is pretty quick at reloading the queue and also firing off the max # of jobs (if at least that many are in the queue) so you don't really have time to get back to see if performance has improved as long as jobs are running.

I think what you want to do is see if there's a time when the slowdown happens and no comksip jobs are running to eliminate that as a cause.

As I said, I had to limit the max # of comskips that can run at a time as well as use the "Run more slowly"/Playnice option or I would get a sluggish menu, and also many times sluggish response during playback of recordings. Sometimes to the point of getting the dreaded spinning circle.

[johnw885]

I think I eliminated the issue, although I'm still not sure what cause it. I ended up creating a symbolic link on the Windows 10 client to the Linux recorded TV folder path.

Ex., mklink /D "/mnt/Videos/RecDir" "//SAGETV/RecDir

It's working just fine now, from what I can tell after a day, it didn't happen once. This suggests that it's an issue with however SageTV streams the Mpeg when the file isn't local. SMB works when the streaming mechanism doesn't.

I have a few theories as to why, but would have to investigate further. It might be UDP packet loss if that's how it's transmitted, due to the HDHomeRunPrime?, or OpenDCT? or the network encoder due to lack of a real video card? Lots of theories...... I might wireshark the network at some point this week to investigate further because I do want to start messing around with the Android Client on the Fire TV, and that can't use SMB. A project for a later date.

Originally Posted by stuckless

When you experience it again... connect to the linux server... (you mentioned that it's a linux vm, so is this a Linux VM running on Windows?) and run the "top" command. Top will show what processes are running, and amount of cpu + memory they are consuming... this might help narrow it down if your think it's the server.

It's running on VMware 6.5, which is basically Linux. Hyper-V isn't bad for Linux, doesn't work well with FreeBSD though, it's all about drivers. I work with Hyper-V, VMware, KVM, XenServer, they all have their ups and downs. I like VMware vSphere the best, but it's expensive. Memory and CPU were topping out at no more than 30 - 50%, it's a beefy Proliant server I got second hand for home testing.

I use commecial detector. You can see a lot of info on what it is doing by going to detailed setup/customize. Then down near the bottom of that menu you'll see "Comskip Playback Options". Click on Modify. A couple up from the bottom of the menu you get will be "Commercial Detector". Click on "Configure". There you'll see all the options for commercial detector and how it runs comskip. You want the advanced options so you're sure to see all the options. I also have the "run more slowly" (i.e. comskips Playnice option) selected on mine and a max of 3 jobs running.

Thanks for this tip, I will be leaving the "playnice" switch on from now on just to be safe. I only run one at a time just because, not sure how the multi threading works or the amount of Disk I/O it uses, maybe I'll knock it up to 2 at once. I ended up eliminating Comskip as my issue by disabling it.

I'm new to SageTV and researching how each piece functions is definitely time consuming. Once I get a little more comfortable with everything I might post a all-in-one noob guide to setup and configuring. I've had to get information or instructions from a lot of different treads coming from MCE. I like the interface much better already though so I really didn't want to go back, thanks for the help! Any tips on OpenDCT with a cable card, or how the network encoder works would be appreciated!