Plugin: Schedules Direct EPG (not for v9)

[rdefino]

I have the same issue I just noticed. EPG runs out tomorrow night.

Anyone have any ideas on what is happening?

[rdefino]

Hi Slugger, any thoughts on what to check?

[rkulagow]

Server-side there haven't been any changes, and other users are continuing to download data.

rdefino, it looks like your last download was 2014-09-25. You're still connecting and getting status and the list of subscribed headends, but it doesn't look like the grabber is actually making a request.

slugger, contact me via PM if you need more server-side logs.

[rdefino]

Originally Posted by rkulagow

Server-side there haven't been any changes, and other users are continuing to download data.

rdefino, it looks like your last download was 2014-09-25. You're still connecting and getting status and the list of subscribed headends, but it doesn't look like the grabber is actually making a request.

slugger, contact me via PM if you need more server-side logs.

Thanks for checking. So for me is the issue serve side?

[rkulagow]

No, I don't believe so. As I said, other users are connecting and pulling data, and your client is connecting and getting status, but not actually requesting data.

[Phydeaux]

Originally Posted by rkulagow

No, I don't believe so. As I said, other users are connecting and pulling data, and your client is connecting and getting status, but not actually requesting data.

Is this the case with me as well, Rob?

RLW

[Slugger]

@Phydeaux: So the logs suggest a corrupted cache file. There were some recently discovered bugs (41 and 42) that can account for corruption of the cache file. Not 100% convinced these are what you're hitting, but it's definitely a possibility.

I'm out of town on personal matters until at least next week. I won't have any time to look into code, etc. Forum responses are also going to be sparse until I return home.

The temp fix: Delete your cache file and force an epg update. There is a snapshot build of the grabber that you could also test.

Replace plugins/sdepg/tools/sdjson.jar with this one (remember to rename the snapshot file to sdjson.jar before copying it in), but I know that the snapshot only addresses one of the bugs so it may not fix anything, but can't hurt to try.

To the others with issues: I need the sdepg log at the very least. Full debug logs are better. As a data point, I just checked my server back home and it's been happily downloading data non-stop with no issues. So it seems the likely cause of everyone's problems is a corrupted cache file, but, as I say, that's only a guess without log files.

With that said, I can't promise when I'll have time to look at the logs before I return home -- evenings are most likely, but be warned I'm now a few time zones closer to the Pacific Ocean so my evenings are later, and by then I'm not really able to sit and read code/logs/etc. But if there's time, I'll try.

[rdefino]

Here is my sdlog file. I can get the debug also.

thank you for any help.

I deleted the cache file and it didn't help.

[rdefino]

Here is the link to the debug files.

https://www.dropbox.com/sh/np0o8g42q...MGQjZJlDa?dl=0


Thanks

[Slugger]

Well the sdepg.log is suggesting there is a problem with the data received from SD:

2014-10-07 11:19:05,989 ERROR [EpgDownloader]: stderr:
Exception in thread "main" java.lang.RuntimeException: org.json.JSONException: JSONObject["stations"] not found.
	at org.schedulesdirect.grabber.Grabber.execute(Grabber.java:869)
	at org.schedulesdirect.grabber.Grabber.main(Grabber.java:914)
Caused by: org.json.JSONException: JSONObject["stations"] not found.
	at org.json.JSONObject.get(JSONObject.java:473)
	at org.json.JSONObject.getJSONArray(JSONObject.java:555)
	at org.schedulesdirect.grabber.Grabber.updateZip(Grabber.java:519)
	at org.schedulesdirect.grabber.Grabber.execute(Grabber.java:807)
	... 1 more

Looks like a lineup map is missing the expected stations field in the response. Hopefully Robert can use that timestamp to search serverside logs? I've downloaded the full debug logs, but can't look at them now. Maybe this evening but no promises. But my initial investigation suggestions a problem upstream.

[rdefino]

Thanks for the update!

[Phydeaux]

Thanks Slugger. Unfortunately deletion of the cache file and forcing another update made no difference. I also tried replacing the sdjson.jar file as suggested to no avail. Here's the log file from the most recent attempt.

RLW


sdepg.log.txt

[Slugger]

Only because the log was so exact at pinpointing this, I'm able to quickly respond:

{"response":"LINEUP_DELETED","code":2106,"serverID":"AWS-SD-web.1","message":"The lineup you requested has been deleted from the server.","datetime":"2014-10-07T15:20:36Z"}

This is the response that is causing rdefino's problem. The lineup you are using has been deleted on the SD side. Not sure why, but that's an issue Robert will need to answer. It's probably just a renamed lineup or something? If you readd your lineups via the web ui it will probably fix the issue.

If someone wanted to open a ticket for me at the sourceforge project site, I could probably better identify this error and alert the Sage user that their lineup has been deleted.

[rdefino]

I went into the Sagetv web UI and removed the line up and re-added it and got the error below immediately.

HTTP ERROR 500

Problem accessing /sage/sdjson/index.gsp. Reason:

    Lineup[USA-MA20419-X]: JSONObject["map"] not found.
Caused by:

org.schedulesdirect.api.exception.JsonEncodingException: Lineup[USA-MA20419-X]: JSONObject["map"] not found.
	at org.schedulesdirect.api.Lineup.fetchDetails(Lineup.java:128)
	at org.schedulesdirect.api.ZipEpgClient.getLineups(ZipEpgClient.java:357)
	at org.schedulesdirect.api.ZipEpgClient$getLineups.call(Unknown Source)
	at sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect.getProviders(EPGImportPluginSchedulesDirect.groovy:125)
	at sage.EPGImportPlugin$getProviders.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at sage.EPGImportPlugin$getProviders.call(Unknown Source)
	at sagex.epg.schedulesdirect.plugin.Plugin.forceEpgRefresh(Plugin.groovy:123)
	at sagex.epg.schedulesdirect.plugin.Plugin$forceEpgRefresh.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
	at SimpleTemplateScript5.run(SimpleTemplateScript5.groovy:138)
	at net.sf.sageplugins.webserver.groovy.templates.IncludeTemplateEngine$SimpleTemplate$1.writeTo(IncludeTemplateEngine.java:120)
	at net.sf.sageplugins.webserver.groovy.servlets.SageTemplateServlet.service(SageTemplateServlet.java:99)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
	at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:879)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)
Caused by: org.json.JSONException: JSONObject["map"] not found.
	at org.json.JSONObject.get(JSONObject.java:423)
	at org.json.JSONObject.getJSONArray(JSONObject.java:499)
	at org.schedulesdirect.api.Lineup.fetchDetails(Lineup.java:119)
	... 34 more
Caused by:

org.json.JSONException: JSONObject["map"] not found.
	at org.json.JSONObject.get(JSONObject.java:423)
	at org.json.JSONObject.getJSONArray(JSONObject.java:499)
	at org.schedulesdirect.api.Lineup.fetchDetails(Lineup.java:119)
	at org.schedulesdirect.api.ZipEpgClient.getLineups(ZipEpgClient.java:357)
	at org.schedulesdirect.api.ZipEpgClient$getLineups.call(Unknown Source)
	at sagex.epg.schedulesdirect.EPGImportPluginSchedulesDirect.getProviders(EPGImportPluginSchedulesDirect.groovy:125)
	at sage.EPGImportPlugin$getProviders.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at sage.EPGImportPlugin$getProviders.call(Unknown Source)
	at sagex.epg.schedulesdirect.plugin.Plugin.forceEpgRefresh(Plugin.groovy:123)
	at sagex.epg.schedulesdirect.plugin.Plugin$forceEpgRefresh.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
	at SimpleTemplateScript5.run(SimpleTemplateScript5.groovy:138)
	at net.sf.sageplugins.webserver.groovy.templates.IncludeTemplateEngine$SimpleTemplate$1.writeTo(IncludeTemplateEngine.java:120)
	at net.sf.sageplugins.webserver.groovy.servlets.SageTemplateServlet.service(SageTemplateServlet.java:99)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
	at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:879)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)
Powered by Jetty://

[Slugger]

Robert will have to have a look server side. You can produce the full debug logs again but I suspect all it's going to tell me is that the SD response is not conforming to spec (the "map" field is not in the response as expected). The debug logs would be able to tell us exactly what is in the response, but I'm hoping Robert can tell that using his server side logs. Again, this looks to be an upstream problem on the SD side.

[Slugger]

Actually that stack says it's using the local cache. So delete the sdjson.epg file first, then go back thru the web ui and try to mod/edit/update your registered lineups.

[rdefino]

You mean the sagetv UI, not the SD ui...right?

[rdefino]

I just removed and added the linue up in the sagetv ui and got this "invalid user" error.


HTTP ERROR 500

Problem accessing /sage/sdjson/index.gsp. Reason:

Error getting lineups! [Invalid user.]
Caused by:

java.io.IOException: Error getting lineups! [Invalid user.]

[Slugger]

Yes, the sage webui. There seems to be some issues upstream. Robert's going to need to help from the other end. I think he gets notified of this thread, but you may want to open a support ticket at SD, which will definitely grab his attention.

[rdefino]

thanks, just opened the ticket. Hopefully it gets fix before prime time tonight.

thanks for the help