Windows Installer

[jusjoken]

Originally Posted by rickgillyon

The 9.1.1.651 files are there now, but are missing the .exe extensions.

ooops. Sorry. My script has been updated and is re-running so the files with the .exe should be there in the next few hours.

k

[jusjoken]

Originally Posted by gordyp

That might not be all they are missing (the .exe part was easy enough to fix, just add it to the downloaded file name) - I had problems with the server download - I was getting a bunch of Java errors (I suspect the java.jar file was missing) and had to roll back to 9.1.0.648 on the server. The 9.1.1.651 client install seems fine, aside from the missing file extension...

Perhaps your download from BinTray corupted it or something else is wrong. I tested the ServerSetup and is works fine on my test system. The missing .exe is being repaired as we speak.

k

[rickgillyon]

Originally Posted by jusjoken

ooops. Sorry. My script has been updated and is re-running so the files with the .exe should be there in the next few hours.

No real need, I'm sure everyone could work out to add the .exe. Thanks for the efforts, much appreciated.

[sflamm]

Out of curiosity- have you measured how much affect string de-duplication has on SageTV? It doesn't seem to me that there would be a lot of shared strings ... just the titles of TV series ... what else? And how much memory were the strings taking before and after? Curious if you have any statistics to share.

[Fuzzy]

He discussed the memory benefits a bit in the github issue: https://github.com/google/sagetv/issues/282

[wayner]

Originally Posted by sflamm

Out of curiosity- have you measured how much affect string de-duplication has on SageTV? It doesn't seem to me that there would be a lot of shared strings ... just the titles of TV series ... what else? And how much memory were the strings taking before and after? Curious if you have any statistics to share.

It would be interesting if people share their experiences with Java heap size usage. This could be a huge benefit to people with 3+ extenders. I went to the unRAID/Linux version to solve this problem myself.

Should we be upgrading our Java to the latest V8, assuming that one is on V8 in the first place?

edit - Maybe not so new as Java 8 Update 20 was released in mid 2014 - there have been something like 50 updates since then.

[sflamm]

He discussed the memory benefits a bit in the github issue: https://github.com/google/sagetv/issues/282

The results are interesting... the source of the duplication was unexpected (to me) but makes a lot of sense.

Looks like the strings that are being repeated are the:

STV files
client configurations files
channel guide
fanart (show descriptions, ....)
etc.

that are being used across clients... most of the string values in these are the same and the GC dedup is pointing all the String classes to the same immutable char array substantially reducing the memory footprint.
This is pretty neat - as writing the SageTV to use shared references across clients normally would be very difficult and this is essentially being done behind the scenes for free.

[sflamm]

Should we be upgrading our Java to the latest V8, assuming that one is on V8 in the first place?

There are multiple threads that point out everything works on Java 8 but a few items related to certain Jetty usages (not core to the product)

[sflamm]

Narflex commented 18 days ago
Nice. There's already a bunch of string.intern() calls in the existing code
for areas where I spotted this as a major problem previously

These string.intern() calls should be removed. They make the deduplication less efficient. (made sense before the built-in feature)

[wayner]

Originally Posted by sflamm

There are multiple threads that point out everything works on Java 8 but a few items related to certain Jetty usages (not core to the product)

I was referring to the fact that this dedup required Java 8 update 20 - but we are WAY beyond that now so I assume that everyone on Java 8 should be good.

[sflamm]

The fanart (images) that are used for each client device are generated by the server. I assume these take a huge amount of the server-side process memory usage.

The images are static... the only processing that is done on them is scaling. And the fanart is shared across all clients.

Would it be possible to store them internally in-memory as a String after being scaled? The data can be streamed from the String to the the clients... benefiting again greatly from the dedup....

[EnterNoEscape]

Originally Posted by sflamm

These string.intern() calls should be removed. They make the deduplication less efficient. (made sense before the built-in feature)

That is likely a bad idea. The deduplication works on the char[] arrays, not the String objects themselves (and they consume space per each allocated String, just some of them now share their char[] array with other String objects). intern() where we know it's a good idea is still better. This just helps us sweep up all of those corner cases and apparently there are a lot.

Originally Posted by wayner

It would be interesting if people share their experiences with Java heap size usage.

You should understand that the numbers I came up with were assembled by nudging the JVM to perform GC and deduplicate. What I have seen when you just let it do this on it's own without any coercion, is it will deduplicate, but it may or may not always clear out all of the char[] arrays at the same time; you won't really see the benefits until you're near the heap limits. So your heap allocation size isn't really going to be a good indicator of anything. What will be a better indicator is how many more miniclients you can use or how much longer everything keeps working before you see OOM errors.

For reference, on a stock SageTV install and ~60MB database, I was able to run multiple miniclients with a max heap size of only 256MB which to me is crazy low. When I looked at the objects on the heap in that configuration, it was clear the deduplication was working very effectively.

Originally Posted by wayner

I was referring to the fact that this dedup required Java 8 update 20 - but we are WAY beyond that now so I assume that everyone on Java 8 should be good.

You are correct that there is a minimum Java 8 version before this was introduced. If you try to run this on a version that's 8, but older than 20 it will just complain about how it doesn't know what to do the parameter. It should not actually prevent the software from working. Also in either case if you're on Java 8, I recommend keeping up with the latest fixes. This is a well established feature, so I don't have any worries about it needing to mature and it's easy to remove from the code if we decide it causing some horrible unforeseen issues.

[wayner]

Originally Posted by EnterNoEscape

You should understand that the numbers I came up with were assembled by nudging the JVM to perform GC and deduplicate. What I have seen when you just let it do this on it's own without any coercion, is it will deduplicate, but it may or may not always clear out all of the char[] arrays at the same time; you won't really see the benefits until you're near the heap limits. So your heap allocation size isn't really going to be a good indicator of anything. What will be a better indicator is how many more miniclients you can use or how much longer everything keeps working before you see OOM errors.

It will be interesting to see how this performs. On Windows with a heap size of 1024 I could run 3-4 miniclients - plus the server client (although I didn't really do anything on the server client). But over time I would start to get warnings about running out of space and and the system would slow down and it sometimes it would just completely crash. There also may have been some other memory leak issues with Windows as this seemed to get worse if the server had been running for weeks.

I now run with a heap size of 3072 and I never even reach 2000 of usage, even when trying by turning on every client I can find. But as you (or someone else mentioned) the Linux version of SageTV seems to be more efficient in memory usage. But I can probably try turning my heap size down to about 1000 and see how it does. Presumably unRAID can use the memory for something else.

[stuckless]

Originally Posted by sflamm

The fanart (images) that are used for each client device are generated by the server. I assume these take a huge amount of the server-side process memory usage.

The images are static... the only processing that is done on them is scaling. And the fanart is shared across all clients.

Would it be possible to store them internally in-memory as a String after being scaled? The data can be streamed from the String to the the clients... benefiting again greatly from the dedup....

No... this would not help and might actually hurt things

Images in the sagetv image system are loaded and cached and evicted from the cache... which is why sometimes you'll see spinning circles when you navigate a very UI intensive menu. If we stored them in strings... that would actually take more memory, since, we'd be keeping the image bytes in the string, but they's have to be uncompressed before you can actually work with them. Today, we only keep the uncompressed images in memory (not the raw bytes from the files). (We also cache the filenames, in phoenix, but that's different than the image contents).

[FreshOne]

Originally Posted by jusjoken

Latest release available

• The following may no longer be required after version 9.0.13.536_R1.010 and above as this may have been fixed.
  • Some users have seen that the installer FAILs to register a number of libraries. This may be related to a missing KB fix needed for VC2015 runtime. IF you run into this then see this KB article and install the related patch and then re-run the installer.

I installed 9.1.1.651 today and had like 7 or 8 files that did not register properly on my Windows 7 SP1 install. After I DLed and ran the KB fix, the Sage Install was finally able to complete successfully.

So the issue hasn't been fixed as far as I can tell.

[PeteCress]

Originally Posted by jusjoken

Is your win7 current with updates? We know this works on window 7....

I am going to make a logical leap and say that the installer probably does not work on XP - mainly because it is failing on my XP box and I do not see mention of any Windows versions except 7 and 10 in the threads I have skimmed so far.

Am I on the right track?

If so, would anybody opine whether I should "Upgrade" my XP box to 7 or 10?

I haven't heard anything good about 10, but it seems to be where things are headed.... OTOH, I know and trust 7.

Also, once I get an acceptable OS on the box, am I going to run into trouble because my SageTV is installed on D and not C: ?

The rationale for D: is that D: is for Data and C: is for System.... so I can image/re-image C: without losing data like Wiz.bin or SageTV.Properties.

[wayner]

Originally Posted by PeteCress

If so, would anybody opine whether I should "Upgrade" my XP box to 7 or 10?

I haven't heard anything good about 10, but it seems to be where things are headed.... OTOH, I know and trust 7.

You asked for opinions - my opinion is that you should upgrade and to Win10 as Win7 is getting pretty long in the tooth. Some folks have complained about having forced reboots on Win10 but I don't think that I have seen that happen.

Other than that I don't see a downside to Win10 and a bunch of things on the upside, such as being newer and having some features that Win7 doesn't have.

For a further opinion - all of the cool kids are moving to unRAID Linux.

[PeteCress]

Originally Posted by wayner

...you should upgrade and to Win10 as Win7 is getting pretty long in the tooth. Some folks have complained about having forced reboots on Win10 but I don't think that I have seen that happen.

Other than that I don't see a downside to Win10 and a bunch of things on the upside, such as being newer and having some features that Win7 doesn't have.

For a further opinion - all of the cool kids are moving to unRAID Linux.

Well, that resolves the 7-vs-10 question for me.

But for a hundred bucks (10 OEM price), maybe I need to re-visit the unRAID solution. Somebody had me going on it a couple weeks ago, but when I tried to create a USB stick to do the deed, I could not make it happen.

A strong appeal to me of the UnRAID solution is the USB stick: it sounds like I can leave the XP install in place and just try UnRAID without losing anything - once I get a USB stick working.... -)

Perhaps more relevant is that it seems like The-Powers-That-Be-SageTV-wise are all using UnRAID....

No problem with Un-RAID and 3-TB drives, right?

[graywolf]

Originally Posted by PeteCress

Also, once I get an acceptable OS on the box, am I going to run into trouble because my SageTV is installed on D and not C: ?

Just when you run the Installer, choose Options before doing the install so you can ensure it is going to the path you want. When I did the V6 > V7 > V9 upgrade on my test image, the V9 Installer did the Default (C:\Program Files (x86)\SageTV) path instead of detecting it was on D: drive.

[waylo]

Originally Posted by PeteCress

Well, that resolves the 7-vs-10 question for me.

But for a hundred bucks (10 OEM price), maybe I need to re-visit the unRAID solution. Somebody had me going on it a couple weeks ago, but when I tried to create a USB stick to do the deed, I could not make it happen.

A strong appeal to me of the UnRAID solution is the USB stick: it sounds like I can leave the XP install in place and just try UnRAID without losing anything - once I get a USB stick working.... -)

Perhaps more relevant is that it seems like The-Powers-That-Be-SageTV-wise are all using UnRAID....

No problem with Un-RAID and 3-TB drives, right?

I think the ONLY reason to stay with Win7 is Windows Media Center. It's been discontinued past Win 8.1 Pro. If you have any need to watch DRM protected channels using a CableCard solution, that's your best bet still... unfortunately.

I myself have my HTPC/server with Win7 because I still want that option...

SageTV isn't the only DVR I have installed, but it is by far the most used.