9.2.1 upgrade no longer updates EPG for some FTA channels

[pendragonsound]

Back in 2017 I was helping SD populate a FTA lineup (ZZZ-FTA.NORTHAMERICA-DEFAULT). There are quirks with this because this lineup is not location or provider specific and it contains fields not normally filled for the usual lineup providers. The applicable thread is here:

https://forums.sagetv.com/forums/showthread.php?t=64669

EnterNoEscape was kind enough to fix the SageTV code so this lineup could be added to Sage for EPG purposes, and sent me a Sage.jar for win32 to be tested. Because it worked so well I simply continued using it since verifying it worked as of October 2017.

A couple of weeks ago I upgraded Sage to 9.2.1, still on win32, as a step towards migrating my Sage system to Linux. Everything appeared to be working fine, but yesterday I noticed some of the FTA channels (mostly PBS related) were no longer updating in the EPG and had less than a week of guide data left, presumably from before my 9.2.1 upgrade. However other FTA channels in the same lineup appear to have been updating, because they now have data that goes into November that likely was not there at the time of my Sage upgrade.

I filed a problem report with SD, and they confirmed their servers had guide data for at least one channel (NHKWRLD) that was failing to update on my Sage system. I manually tried forcing guide updates on this channel by deleting the channel in Sage and later adding it back in, but nothing new was added.

Out of curiosity and after backing up the critical files, I restored the files necessary to run the 'test' version of Sage from October 2017. The EPG immediately was extended for all the channels that had not been updating, including NHKWRLD and all of the PBS channels. Is there something that I am missing, and/or is there anything I can do to help get this fixed? Thanks!

[EnterNoEscape]

I'm fairly sure any changes in the jar I gave you were merged with the main project. If you can enable JSON logging and post the resultant logs (including the SageTV logs) along with a few station ID's that aren't coming in, I'll see if I can determine where the issue is.

[pendragonsound]

My recollection was the change you made in 2017 was to allow me to add this lineup to Sage, which is a different problem than I'm now experiencing. I'll plan to switch back to 9.2.1 tomorrow and turn on the JSON logging as you suggest, but it might take a few days before it becomes apparent if any channels are not updating. I'll get back as soon as I have something. Thanks!

[pendragonsound]

When I switched over to 9.2.1 today I figured I'd wait a couple of days before turning on the JSON debugging, because by then I would know which channels were populating new EPG entries and which ones weren't. However I just looked at the regular Sage log and noticed the following:

Wed 10/16 21:56:03.159 [EPG@13a9407] EPG Expanding FTA - North America -
North America (sdepg) at Wed 10/16 21:56:03.159
Wed 10/16 21:56:03.586 [EPG@13a9407] SDEPG Exception thrown:
java.lang.NumberFormatException: For input string: "11916500000"
Wed 10/16 21:56:03.587 [EPG@13a9407]
sage.epg.sd.gson.JsonSyntaxException: java.lang.NumberFormatException:
For input string: "11916500000"
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.TypeAdapters$7.read(TypeAdapters.java:245)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.TypeAdapters$7.read(TypeAdapters.java:235)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:72)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.Gson.fromJson(Gson.java:889)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.Gson.fromJson(Gson.java:954)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.TreeTypeAdapter$GsonContextImpl.deserialize(TreeTypeAdapter.java:163)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.json.map.SDLineupMapDeserializer.deserialize(SDLineupMapDeserializer.java:58)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.json.map.SDLineupMapDeserializer.deserialize(SDLineupMapDeserializer.java:30)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:70)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.Gson.fromJson(Gson.java:889)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.gson.Gson.fromJson(Gson.java:827)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.SDSession.getAuthJson(SDSession.java:692)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.SDSession.getLineup(SDSession.java:866)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.epg.sd.SDRipper.extractGuide(SDRipper.java:1259)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
sage.EPGDataSource.expand(EPGDataSource.java:709)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at sage.EPG.run(EPG.java:773)
Wed 10/16 21:56:03.587 [EPG@13a9407]     at java.lang.Thread.run(Unknown
Source)
Wed 10/16 21:56:03.587 [EPG@13a9407] Caused by:
java.lang.NumberFormatException: For input string: "11916500000"
Wed 10/16 21:56:03.587 [EPG@13a9407]     at
java.lang.NumberFormatException.forInputString(Unknown Source)
Wed 10/16 21:56:03.588 [EPG@13a9407]     at
java.lang.Integer.parseInt(Unknown Source)
Wed 10/16 21:56:03.588 [EPG@13a9407]     at
java.lang.Integer.parseInt(Unknown Source)
Wed 10/16 21:56:03.588 [EPG@13a9407]     at
sage.epg.sd.gson.JsonPrimitive.getAsInt(JsonPrimitive.java:260)
Wed 10/16 21:56:03.588 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.JsonTreeReader.nextInt(JsonTreeReader.java:244)
Wed 10/16 21:56:03.588 [EPG@13a9407]     at
sage.epg.sd.gson.internal.bind.TypeAdapters$7.read(TypeAdapters.java:243)
Wed 10/16 21:56:03.588 [EPG@13a9407]     ... 19 more
Wed 10/16 21:56:03.588 [EPG@13a9407] ERROR Updating EPG Source FTA -
North America - North America (sdepg)
Wed 10/16 21:56:03.588 [EPG@13a9407] ErrorText:expand called on FTA -
North America - North America (sdepg) at Wed 10/16 21:56:03.158
expandedUntil=Wed 10/16 21:55:02.712 scannedUntil=Sun 8/17 0:12:55.807
EPG Expanding FTA - North America - North America (sdepg) at Wed 10/16
21:56:03.159

This may be a related problem to what happened before, because that field looks like the frequency (in Hz) of a satellite transponder for one of the channels in the lineup. I'll turn on the JSON logging once people here stop watching TV, but that will likely be tomorrow.

[pendragonsound]

Sent you a PM with the logs you requested, along with a partial list of FTA channels that do not update and some that don't.

[EnterNoEscape]

From a first look, it seems we have the issue from the previous thread happening. I'll try to dive into it a little more tonight and have something for you to test out this week.

[EnterNoEscape]

The issue is that the frequency is a signed integer (deja vu) and that number exceeds the max value for a signed integer. I guess the change never made it into the code base which is kind of surprising to me, but not impossible. I want to verify how/if SageTV cares about this number and if being changed to a long type would be a problem (I think the answer is no) before I submit anything to the main project.

[pendragonsound]

Thanks for the initial look! There's no rush because it appears I can fall back to the 2017 test version in the meantime. Once I have things stabilized on my ancient win32 server I'll be also testing its Linux replacement in parallel, so I may need a few pointers on how to get a Sage jar for that.

[wnjj]

Originally Posted by pendragonsound

Thanks for the initial look! There's no rush because it appears I can fall back to the 2017 test version in the meantime. Once I have things stabilized on my ancient win32 server I'll be also testing its Linux replacement in parallel, so I may need a few pointers on how to get a Sage jar for that.

The jar is the same across OS's.

[EnterNoEscape]

If anyone is interested, nothing currently depends on the variable mentioned above. I've compiled a fixed copy of Sage.jar and pendragonsound will be testing it before I commit the fix to the main project.