SageTV Service using 1.18GB of mem?

[wayner]

Recently my Sage system is running slightly slower than it used to. I restarted the service a few hours ago and it is already up to 1.18GB of memory used. Note that I do have lots of extenders and since the reboot at least four different HD-200s have connected to the server.

Is this normal or do I have a mem leak somewhere?

[panteragstk]

I'm at 709mb without any extenders so I guess it's normal. I think it has more to do with the plugins you have installed as well as how many extenders connected. Mine has started going a bit slower as well, but I haven't had the time to figure out why. I did change the hard drive in my server to a slower one when I rebuilt it, so I'm thinking that is the issue. Once I put the faster drive in I'll post my results.

[Rezolution]

I have my Java heap set to 1Gig.

I have 4 extenders in the house. When SageTV is idle and sitting in my tray, it always sits at 1.1 ish Gig. When I turn something on to watch a show, it shrinks down in size to around 500M.

I'm not sure why, but that's pretty normal behavior (for me at least) running v6.6.

[Skybolt]

I am not sure Wayner asking about the Jave Heap size, but the actual memory used by the SageTvService.exe. If I am wrong then I appolgize.

I set my Java heap to 1024MB w/6 clients on average.
But my SageTvService.exe peaked at 6,388K and average is 2,500K.

If your SageTvService.exe is showing >1gig mem usage in task manager then something is definetly wrong.

-Skybolt

[wayner]

I have been getting out of mem errors on a regular and the SageTV service crashing. My JVM Heap Size is 372MB/575MB/1038MB.

I don't know what can be causing this but I have tried disabling a number of plugins. I have recently added an extedner so I now have seven extenders and I have relatives over so in the last few days there has often been 4 or 5 extenders running simultaneously.

The only recent changes to my system, other than adding an extender, have been upgrading a few plugins like SRE. But there haven't even been many updates to apply in recent months.

When I restart the SageTVService it will ramp up to over 1 GB used within a minute, even if no extenders are being used. However by default Sage seems to turn on all extenders upon the service being restarted.

Maybe I should try unplugging a few extenders to see if that does anything?

[Slugger]

I'd be surprised if a server could host that many extenders. I don't know the design/implementation details of the core extender functionality - and we never will, of course - but based on my experience, connecting a single extender to the server chews up a quick 150-200MB of heap (stock STV + Phoenix fanart on one, just stock STV on the other). Connecting a second extender seems to chew up another 150MB or so of heap. So 7 x 150 = 1050, which would exceed your JVM's heap space. And that's the low end. If you're running a graphically intense STV on each extender - like Phoenix or Diamond, etc. then I'm quite sure each extender is chewing up way more than 150MB of heap per connection.

I'm sure the core extender implementation finds ways to share objects in the heap, when possible, to lower heap requirements. But do the plugins/alternate STVs? If not, then it's no real surprise 4+ simultaneous extender connections causes an OOM exception in your JVM.

[wayner]

Thanks Slugger - I have been running 5 extenders for years - first under SageMC on both v6 and v7 for the last year or so using Diamond on v7. I added a sixth extender last spring and still had no probs. it appears that extender # 7 is when the problems started to appear - either that or one of the recently upgraded plugins. And other than you and Tmiranda there isn't much updating of apps happening.

I will try unplugging a few extenders and see if that changes behavior.

I was worried that the problem was something else so I started disabling someof the recently updated stuff like SRE, Dynamic skip times, SRE UI, etc.

[firenice]

I am running WHS v1 with 2 HD300 extenders and 1 client. When idle and no extenders running the SageTVService sets aound 360MB when both extenders are going it is around 520MB. The client didn't seem to add much to this. You could possibly add another 500MB with an additional 5 extenders running though killing the heap. The memory isn't released right away so if extenders are turn off and on quite a bit this will affect the heap also.

Good Luck
Mike

[Fonceur]

Originally Posted by wayner

However by default Sage seems to turn on all extenders upon the service being restarted.

No, that's the SageTCPServer plugin, unless the Auto-reboot option is disabled, which leads to other issues...

[wayner]

@fonceur - So if I am using SageTCPServer there is no way to prevent all of my extenders from turning on when the service is restarted other than unplugging them? Is it possible for you to change this in SageTCPServer?

[Fonceur]

Originally Posted by wayner

@fonceur - So if I am using SageTCPServer there is no way to prevent all of my extenders from turning on when the service is restarted other than unplugging them?

As I said: unless the Auto-reboot option is disabled. You do that through the SageTCPServer Configuration...

[wayner]

But you said that leads to other issues - what are the other issues? I want to make sure that the cure is not worse than the disease.

[Fonceur]

Originally Posted by wayner

But you said that leads to other issues - what are the other issues? I want to make sure that the cure is not worse than the disease.

The other issues are the reasons why I added that option in the first place... For one thing, SageTV isn't aware of the extenders, so you couldn't switch them on with T*SageTV or other third party clients (like the CQC driver), unless you used the IR remote once since the restart. I know that people have requested the auto-reboot before, you'd have to search old threads to see why...

[wayner]

When I get a chance to reboot I will see if the memory usage is different with this option on or off. For my usage I don't mind turning on the extenders via IR. I often do maintenance at night when the rest of the family is sleeping and I didn't like the Sage reset turning on the LEDs on extenders in my bedrooms, especially the bright blue of an HD 300.

[wayner]

It appears that the problem was caused by too many extenders. SageTCP Server was turning all 7 of them on automatically upon a restart of the Sage service. This meant that the SageTVService process was using up 1.2GB of memory within a few minutes of being turned on.

I shut off this option and rebooted the PC. Without any extenders connected the service was using 259M as I added extenders here is the memory used:

Extenders - Memory
0 - 259M
1 - 471M
2 - 654M
3 - 856M
4 - 974M
5 - 1064M

So the first three extenders seem to use 200M each. The fourth and fifth used about 100M each. I couldn't turn on the others due to the time of night and not wanting to disturb sleeping people but it looks pretty obvious that 7 extenders is going to cause an overload.

Note that shutting down extenders does not release memory. The memory usage didn't change as I powered them down. But it looks like they don't actually power down - they just go into standby mode as the extenders were all still visible in the Sage Web home page.

So the moral of the story is that if you use more than 5 or 6 extenders you are asking for trouble.

Does anyone know of a way to really shut off an extender and releasing its memory usage? Would this require a power down?

[Slugger]

Depends how you're checking your memory? Once the JVM grabs memory, it doesn't give it back. So if you're checking via Windows task manager, the process isn't going to appear to release any memory. Internally, the JVM will release the memory for reuse as the app continues to run.

If you're looking at the JVM mem stats in Sage, there's a few other things going on. Again, depending on which plugins/STVs you're running, you may run into problems.

I don't remember which plugin as this was back in v6 for me, but I was running a plugin that didn't play nice and simply held on to references of something from the core. Whatever it was doing, it caused the core not to ever release the heap being used for that UI connection. The end result was, after enough extender restarts I'd run out of memory because no connections were ever being fully released. This sounds like it could be something you're facing because, in theory, once you disconnect an extender it should release all objects from the heap. Of course, the JVM may not garbage collect immediately either - though it should if you are on the verge of exhausting your heap. But if there's some reason it can't collect (i.e. a bad behaving plugin) then you're out of luck - you'll eventually run out of memory. Then again, it may not be a bad plugin at all - if you're running Phoenix or Diamond then you're just likely using way more memory than you think. Just because connecting chews up 200MB doesn't mean you're not using way more as you navigate menus and load up your 1080p fanart images. I could go on with more scenarios where fanart chews up mem that doesn't ever get released and it wouldn't even be fanart's fault, but I think you get the idea.

At the end of the day, with stock STV only, a JVM with 1GB of heap would struggle, imho, to host more than 4 simultaneous extenders/placeshifters. It might be able to do it, but you're always going to be on the brink of an OOM exception. Throw in a prettier/graphically intense STV and 4 is definitely your max. Your saving grace would have been a port of Sage to the 64bit JVM where your only limitation would have been how much RAM you were willing to shove into your system, but obviously that's not going to happen now therefore you're probably limited to about 4 (maybe 5, if you're lucky) simultaneous extender connections to the server.

[wayner]

I was measuring usage via the Windows Task Manager. As I mentioned above I have run 5 extenders with no problems for a long time and I have had a sixth extender since last May. Using more than three extenders simultaneously was rare and I rarely had any OOM issues.

But a few weeks ago I added a seventh extender but I rarely used it as it was in a spare bedroom. I had relatives visiting over the holidays and they were using that extender and given all the people in the house it was not unusual for 4-5 extenders to be running at once.