SageTV JVM RAM - Windows

[nycjoe]

This has been asked a million times, but I just can't find a clear thread that gives a straight answer... What's a good value to set java heap to?
I recently moved from sagetv Linux to Windows and it has been crashing on me non stop. I have plenty of RAM (16GB/win8.1) so I should have any contiguous memory amount available (all I have is a 4gb VM running on this host).

OOM crashes also seem to take down the Hauppauge tuner service which then also needs to be restarted (is that normal?).

Also, is there a place (and a reason) to set Xms? I believe I used to have that set very high as well.

I think I had it set to 1536mb on my previous machine, is that a good #?
https://screencloud.net/v/rNvh

BTW part of the reason I wanted to move Linux->Windows was because I was considering attaching the server to the L/R TV to replace an extender (I have vague memories of video skipping being zippier that way). BOY was that a flawed notion. The HD300's are unmatched, unparalleled, (unbowed, unbent, unbroken).

[will]

Originally Posted by nycjoe

I recently moved from sagetv Linux to Windows and it has been crashing on me non stop. I have plenty of RAM (16GB/win8.1) so I should have any contiguous memory amount available (all I have is a 4gb VM running on this host).

First off, SageTV is a 32bit application which means it can't address more than 2gigs of RAM - so all that extra RAM can't be used for SageTV...

Now there are ways to get a 32bit application to address up to 3gigs of RAM on certain 32bit Window Server versions (using physical address extensions for remapping memory). I wrote about this and SageTV/server stability almost six years ago.

With that being said, in theory, all 32bit applications on 32bit OSs should be able to address 3gigs of RAM if they are compiled a certain way (though the max appears to be 2gig for SageTV) and all 32bit applications on a 64bit OSs (like you are running) should be able to address 4gigs of RAM (if they were compiled a certain way). However, I still don't think SageTV can use more than 2gigs of RAM (I haven't verified this though).

I would recomend you play around with SageTV and try setting your heap size to 1000 (4096mb) and see if that makes a difference.

For the record, I have my HEAP set to 400 (1024mb) and SageTV runs very stable for me - six years running with a huge wiz.bin file.

I suggest you check out my post in the link above because it addresses the instability I had with my Hauppauge tuner and trying to use more than 2gigs of RAM for SageTV. I'm not sure how relavent my fix is for Windows 7/8/10 but it did take a very unstable SageTV system that crashed every few hours to being rock solid. Perhaps some of the things discussed in that forum can lead to some helpful Google searches.

Good luck getting your system stable.

[reggie14]

I've been under the impression that Sage won't use more than 1024MB.

[Fuzzy]

It's not that sage won't 'use' more than 1024, but set higher than that runs the risk of instability, because java's somewhat lazy garbage collection tends to lead towards out of memory errors if you go much higher than that. There's hopes that some architectural changes going forward will allow more memory use by splitting each extender instance to it's own JVM process. Apparently, that's how google is doing it for the Google Fiber, and it works very well. If you don't have extenders, and all your UI's are clients (which WOULD be in their own JVM's on the client machine), 1024MB is plenty for any sage server.

The 'simple fix' everyone seems to think shifting to 64-bit is turns out to not be so simple.

[nycjoe]

Is anyone using sagetv with java 1.8?

Originally Posted by will

For the record, I have my HEAP set to 400 (1024mb) and SageTV runs very stable for me - six years running with a huge wiz.bin file.

How large is a huge wiz.bin file? Mine is 40mb, which sounds small to me.

Originally Posted by Fuzzy

It's not that sage won't 'use' more than 1024, but set higher than that runs the risk of instability, because java's somewhat lazy garbage collection tends to lead towards out of memory errors if you go much higher than that.

IIRC, when setting up JVM, it's important to set xms and xmx not too far apart, in order to avoid aggressive garbage collection that never finishes or causes the process to hang etc. It's been about 10 years since I've really worked with java so I'm def rusty, though.

Looking at my hotspot log shows this:
jvm_args: -Djava.library.path=C:\Program Files (x86)\SageTV\SageTV\; -Xmx1536m vfprintf -Xms24m -server -verbose:gc -XX:+ForceTimeHighResolution

Which is a low xms value, does anyone know how to change that? I can't even figure out how to show the current value in windows (what's the win equivalent of 'ps'???)

This is what I had set previously:
$ grep JAVAMEM startsagecore
#JAVAMEM=-Xmx384m
JAVAMEM="-Xmx1536m -Xms1024m"
java -Djava.awt.headless=$HEADLESS -DUSERKEY=$userkey $JAVAMEM -XX:+UseAdaptiveSizePolicy -XX:MaxGCPauseMillis=25 -XX:GCTimeRatio=24 -XX:ThreadPriorityPolicy=1 $JAVAOPTS -cp Sage.jar:.:/:$(echo JARs/*.jar | sed 's/ **/:/g') SecuredLoader 0 0 x "sagetv Sage.properties" <&- &

I can share a pastebin of my hotspot if anyone is interested... Not sure if there are license keys in there or what.

[fidget]

Originally Posted by Fuzzy

It's not that sage won't 'use' more than 1024, but set higher than that runs the risk of instability, because java's somewhat lazy garbage collection tends to lead towards out of memory errors if you go much higher than that. There's hopes that some architectural changes going forward will allow more memory use by splitting each extender instance to it's own JVM process. Apparently, that's how google is doing it for the Google Fiber, and it works very well. If you don't have extenders, and all your UI's are clients (which WOULD be in their own JVM's on the client machine), 1024MB is plenty for any sage server.

The 'simple fix' everyone seems to think shifting to 64-bit is turns out to not be so simple.

That would explain some of the strange memory problems I had been seeing. Fortunately, updating to Java 8 seemed to have fixed them.

[will]

Originally Posted by nycjoe

How large is a huge wiz.bin file? Mine is 40mb, which sounds small to me.

"Huge" relative to what your wiz.bin file starts out at (around 400k with a fresh v7 install - almost a 10,000% increase).

Mine is ~40mb after almost six years of pretty heavy use. It seems like a large wiz.bin file maxes out between ~40mb to ~50mb based on a recent discussion. I think the reason why the wiz.bin stopped growing beyond 50mb is because STV started embedding meta data in v7.

40mb is nothing compared to a single recording but it seems like a pretty good size to me for a flat-file database.

[stanger89]

Originally Posted by will

Mine is ~40mb after almost six years of pretty heavy use. It seems like a large wiz.bin file maxes out between ~40mb to ~50mb based on a recent discussion. I think the reason why the wiz.bin stopped growing beyond 50mb is because STV started embedding meta data in v7.

That info is all still in the wiz.bin, it's just embedded in the file to allow "recovery" of metadata if the files are moved.

[Taddeusz]

Maximum heap size varies slightly from system to system. You can test for your maximum heap size by dropping to a command prompt and entering something like the following:

javaw -Xmx1448M

BTW, that is the maximum heap size for my SageTV machine. I currently have mine set to just 1024 (0x400).

[phelme]

the Wiz.bin size has never been the issue for me, it's the plug-ins.

I gave up using Phoenix fanart/images because I have so many recordings that using artwork on my HD300 *always* causes the server heap to creep to max. This makes the interface sloth-slow within 2 days unless the server is restarted.

no offense to sloths intended.

[wayner]

I thought the max you could use was 1024 (decimal) and I have been using that for years. I have 9 extenders connected to the system and I have to make sure that extenders are shut off when not in use or I will run out of JVM that will cause Sage to crash or slow to a crawl.

[Taddeusz]

Originally Posted by wayner

I thought the max you could use was 1024 (decimal) and I have been using that for years. I have 9 extenders connected to the system and I have to make sure that extenders are shut off when not in use or I will run out of JVM that will cause Sage to crash or slow to a crawl.

I've had mine set higher before without issue. I don't remember the reason I have mine set at 1024. Was not because of instability.

[BobPhoenix]

Originally Posted by Taddeusz

I've had mine set higher before without issue. I don't remember the reason I have mine set at 1024. Was not because of instability.

I know in the past when I tried to set it higher the service would just crash because I didn't have that much contiguous memory. But if I remember correctly that was with 32 bit XP.

[nycjoe]

Originally Posted by nycjoe

This has been asked a million times, but I just can't find a clear thread that gives a straight answer... What's a good value to set java heap to?
I recently moved from sagetv Linux to Windows and it has been crashing on me non stop. I have plenty of RAM (16GB/win8.1) so I should have any contiguous memory amount available (all I have is a 4gb VM running on this host).

If anyone is interested, I broke down and moves sagetv back to Linux, which is running fine with 1.5gb xmx. I now have a weird setup where my nuc is running Windows (8), which allows the family to switch input and use the windows computer easily for whatever reason. Inside it, there is a virtualbox Linux vm running sagetv (I will never update or touch this server for any reason) that smb mounts a cifs share from windows for recordings, and I'm running another virtualbox Linux vm for external ssh/http access and stuff like comskip etc