SD: "NO DATA" issue on select channels

[MacDaddy]

I'm using a paid SD lineup for Comcast digital channels in 77096 (Houston). The vast majority of the channels load data correctly. I see "NO DATA" for WEATHHD 627, WGNA 654, TRAVHD 662, and IDHD 713 after multiple refreshes. I posted a ticket to SD and they show that there is data available. Here is a snippet from the log file that leads me to believe there is some kind of parsing error:

Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.stream.JsonReader.beginArray(JsonReader.java:350)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:70)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	... 14 more
Sun 4/16 12:25:39.221 [EPG@360bb7b4] ERROR Updating EPG Source Comcast - Digital - Houston (sdepg)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] ErrorText:expand called on Comcast - Digital - Houston (sdepg) at Sun 4/16 12:25:07.343 expandedUntil=Sun 4/16 12:24:02.229 scannedUntil=Sun 8/17 1:12:55.807
EPG Expanding Comcast - Digital - Houston (sdepg) at Sun 4/16 12:25:07.343

The last log0 file is attached.

Any help to resolve would be most appreciated.

* Thread split to its own topic and moved *

[Slugger]

Doesn't look like you're using the plugin that this thread was originally for? That partial stacktrace looks like you're using the native SD support for v9? If so, you should probably head over to another thread or maybe start a new one in the EPG area? I don't think you're gonig to get many looks here (as this thread referred to an older plugin that is no longer the recommended way to access SD epg data in v9).

[EnterNoEscape]

Originally Posted by MacDaddy

I'm using a paid SD lineup for Comcast digital channels in 77096 (Houston). The vast majority of the channels load data correctly. I see "NO DATA" for WEATHHD 627, WGNA 654, TRAVHD 662, and IDHD 713 after multiple refreshes. I posted a ticket to SD and they show that there is data available. Here is a snippet from the log file that leads me to believe there is some kind of parsing error:

Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.stream.JsonReader.beginArray(JsonReader.java:350)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:70)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	at sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] 	... 14 more
Sun 4/16 12:25:39.221 [EPG@360bb7b4] ERROR Updating EPG Source Comcast - Digital - Houston (sdepg)
Sun 4/16 12:25:39.221 [EPG@360bb7b4] ErrorText:expand called on Comcast - Digital - Houston (sdepg) at Sun 4/16 12:25:07.343 expandedUntil=Sun 4/16 12:24:02.229 scannedUntil=Sun 8/17 1:12:55.807
EPG Expanding Comcast - Digital - Houston (sdepg) at Sun 4/16 12:25:07.343

The last log0 file is attached.

Any help to resolve would be most appreciated.

This is the wrong place for the core SD support, but the stack trace makes me think Schedules Direct is sending us some invalid JSON. You should probably start a new thread, enable JSON logging, do an EPG update, then attach the sd_epg.log log file. I can then see if we can accommodate whatever we discover is being returned.

Edit: I just saw this parsing issue myself. I'll be submitting a ticket to Schedules Direct because the format being returned is not what the API says should be returned and I have not received any notices that the API has been changed in this regard.

Bad formatting:

{
  "programID": "MV003820310000",
  "airDateTime": "2017-04-29T20:00:00Z",
  "duration": 9000,
  "md5": "Sjk+Bn95ADoEHxGFMoUSHg",
  "audioProperties": [
    "cc",
    "dvs"
  ],
  "videoProperties": {
    "0": "hdtv",
    "2": "letterbox"
  },
  "ratings": [
    {
      "body": "USA Parental Rating",
      "code": "TVPG"
    }
  ]
}

Expected formatting:

{
  "programID": "MV005521880000",
  "airDateTime": "2017-04-29T22:30:00Z",
  "duration": 9000,
  "md5": "Fe7weiuYHZXKHVTRlndq/Q",
  "audioProperties": [
    "cc"
  ],
  "videoProperties": [
    "HD 1080i",
    "hdtv"
  ],
  "ratings": [
    {
      "body": "USA Parental Rating",
      "code": "TVPG"
    }
  ]
}

[EnterNoEscape]

It looks like the problem I described is already being discussed:

http://forums.schedulesdirect.org/vi...hp?f=15&t=2902

[MacDaddy]

Sorry folks, my bad. I've been away for a while and just now getting back in to things again. I was negligent in searching for the correct SD feature.

As it appears this is being worked, I'll stand down (unless you need those JSON logs), I'll follow progress over in the correct thread.

[EnterNoEscape]

Originally Posted by MacDaddy

Sorry folks, my bad. I've been away for a while and just now getting back in to things again. I was negligent in searching for the correct SD feature.

As it appears this is being worked, I'll stand down (unless you need those JSON logs), I'll follow progress over in the correct thread.

I wouldn't expect most people to catch the root cause of this problem.

I did submit a pull request to the SageTV project on GitHub with a workaround so that while Schedules Direct are working on this issue, we will stop getting that error. If I'm looking at things correctly, it's possible we'll see this error randomly for the next 23 days.

[rkulagow]

Once the root cause of the issue was identified on our side, all of the schedules were re-generated, so that would have created new MD5 values. If the grabber is using the MD5 of the schedule to determine which schedules need updates, then it should automatically self-heal on the next grabber run.

[EnterNoEscape]

Originally Posted by rkulagow

Once the root cause of the issue was identified on our side, all of the schedules were re-generated, so that would have created new MD5 values. If the grabber is using the MD5 of the schedule to determine which schedules need updates, then it should automatically self-heal on the next grabber run.

How long should that have taken? I saw bad data still coming in as of late last night.

[rkulagow]

Everything was done regenerating around 1130P, US Central time.

[EnterNoEscape]

Originally Posted by rkulagow

Everything was done regenerating around 1130P, US Central time.

Thanks. The data appears to be normal again today.

[MacDaddy]

I can confirm that WGNA 654, TRAVHD 662, and IDHD are showing good data. Unfortunately WEATHHD 627 is still showing "NO DATA".

[EnterNoEscape]

Originally Posted by MacDaddy

I can confirm that WGNA 654, TRAVHD 662, and IDHD are showing good data. Unfortunately WEATHHD 627 is still showing "NO DATA".

Try deleting sdmd5sched, then force another EPG update. 627 should have been fine too since the integrated support doesn't save the hashes unless the entire update is successful, but I've noticed what appears to be an issue with the hallmarks of a race condition in SageTV. It happens when new channels are added and seems to prevent the imported data from actually being imported on the first pass. I've seen it happen with the original EPG service too, so I know this isn't a problem with SD. If for some reason that channel was seen as a new channel, my suggestion will work.

[MacDaddy]

Thanks for the suggestion. I deleted sdmd5sched from appdata/sagetv/server, forced an EPG update in the web interface and the schedule data for all channels is working.