Schedules Direct EPG Plugin PoC Completed

[Slugger]

Just to ease any anxiety, I've completed a PoC for a Schedules Direct EPG plugin. It meets all of my goals:

1) Can be installed as a plugin
2) Once installed and configured, just restart Sage and forget about it; Sage operates just as if you were using the Sage EPG source
3) No external tools, no XMLTV, no nothing external required - install the plugin, configure it with your SD id/pwd, go configure your inputs to use the new EPG soruce and forget about it
4) Support for both Linux and Windows

It's definitely rough around the edges and I see a few obvious bugs in it, but it works. I'm using it as the EPG source on my Linux test box and it works absolutely fine. So this proves that in case of doom's day scenario, there is a quick, simple to use EPG replacement available. And since we've been told we have at least a year before it should be necessary, that should give me plenty of time to work out the kinks.

The only issue with this particular plugin is that I can't release it without an exemption from Schedules Direct. However, when the need for this plugin arises then the exemption required shouldn't be an issue. Again, it seems we have ~12 months to worry about that.

Screenshots from Placeshifter connected to my Linux server. This plugin will work on both Linux and Windows Sage servers.

[Enormous]

Excellent, excellent work. Thank you!

[KeithAbbott]

Great news! A couple questions, though.

1. Not that I would need anything more, but Schedules Direct looks to be just US and Canada guide data. True?
2. Is the data available similar to Zap2it, as far as the look-ahead window? Two weeks?

[hugenbdd]

Originally Posted by Slugger

Just to ease any anxiety, I've completed a PoC for a Schedules Direct EPG plugin. It meets all of my goals:

1) Can be installed as a plugin
2) Once installed and configured, just restart Sage and forget about it; Sage operates just as if you were using the Sage EPG source
3) No external tools, no XMLTV, no nothing external required - install the plugin, configure it with your SD id/pwd, go configure your inputs to use the new EPG soruce and forget about it
4) Support for both Linux and Windows

It's definitely rough around the edges and I see a few obvious bugs in it, but it works. I'm using it as the EPG source on my Linux test box and it works absolutely fine. So this proves that in case of doom's day scenario, there is a quick, simple to use EPG replacement available. And since we've been told we have at least a year before it should be necessary, that should give me plenty of time to work out the kinks.

The only issue with this particular plugin is that I can't release it without an exemption from Schedules Direct. However, when the need for this plugin arises then the exemption required shouldn't be an issue. Again, it seems we have ~12 months to worry about that.

Screenshots from Placeshifter connected to my Linux server. This plugin will work on both Linux and Windows Sage servers.

WOW!!! That is great.
I have a few questions.

1.) Can you explain the exemption needed before release?
2.) Will this maintain the "show id", so that Sage will know weather we have watched a repeat? (Sorry, can't remember the exact name/definition of this feature).

Thanks
Dave

[graywolf]

Wonderful News. Great job as always Slugger

[stuckless]

Good to know that there will be something in place.

[stuckless]

It would nice to have a gap analysis of what is different between sagetv and SD. ie, things like, is there season/episode information, etc, unique seriesid #, does sagetv get an "externalid" of "EPXXX", "SPXXX", "MVXXX", etc.

Phoenix uses this information as part of the "scraping" process, so it would be nice to know if that will need to change.

[kmp14]

It is simple. Slugger.....you are awesome.

[Slugger]

Originally Posted by KeithAbbott

Great news! A couple questions, though.

1. Not that I would need anything more, but Schedules Direct looks to be just US and Canada guide data. True?
2. Is the data available similar to Zap2it, as far as the look-ahead window? Two weeks?

The source of the data is TMS (Tribune), which is the same source as Z2I (all the same company). There is 14 days worth of data available (actually looks like 13 and a bit, but let's call it 14).

EDIT: Yes, SD provides listings for US, Canada, a few select US territories and perhaps a couple other locations. Mainly, though, just US and Canada.

Originally Posted by hugenbdd

WOW!!! That is great.
I have a few questions.

1.) Can you explain the exemption needed before release?
2.) Will this maintain the "show id", so that Sage will know weather we have watched a repeat? (Sorry, can't remember the exact name/definition of this feature).

Thanks
Dave

1) Schedules Direct is a non-profit client of TMS with a very specific contract. I'm not a lawyer and don't know all the specifics, but SD is only able to provide EPG data to individual users who will not in turn feed the data to commercial applications. SageTV is considered a commercial application.

Furthermore, software provided for public release that use this data must also be free of cost and/or open source.

This plugin will be open source so that's not an issue. The only issue is that the plugin feeds data into SageTV, which is still considered a commercial app. However, when the need arises for this plugin it would be presumed that SageTV is no longer a viable commercial app and therefore, I'm told, that getting an exemption to release it publicly won't be an issue. Just in case that exemption doesn't come through, I have a backup plan. Let's just say, either way, EPG data will be available, though the backup plan isn't nearly as simple to use as this solution is.

2) Yes. Since Schedules Direct gets its data from TMS, which is the same source as Sage's EPG, the Show IDs will remain consistent.

Originally Posted by stuckless

It would nice to have a gap analysis of what is different between sagetv and SD. ie, things like, is there season/episode information, etc, unique seriesid #, does sagetv get an "externalid" of "EPXXX", "SPXXX", "MVXXX", etc.

Phoenix uses this information as part of the "scraping" process, so it would be nice to know if that will need to change.

Some quick things I noticed while writing the plugin:

• There is no season/episode info. Either SD doesn't pay TMS for this extra info in their feed or Sage pulls/deduces this info from elsewhere and adds it to their EPG data server.
• SD does not provide the 'Live' flag for shows airing live.
• SD does not provide many of the extra program details such as 'Season/Series Premiere/Finale', DD5.1, etc. Some are provided like HDTV, Stereo, CC.
• The unique series ID is in the feed, I believe, but I'd have to double check. I know I'm not bringing it in right now. Not a big deal to add it later.
• The external show IDs are exactly the same as those we get today from Sage's server (EP, SH, SP, MV)

[kricker]

Schedules Direct costs $20 a year is that right? Is that the kind of subscription we will need with them to use this plugin?

[sumrtym]

I'm really not liking the lack of Season / Episode data. To me, that's a must have.

[tvmaster2]

Originally Posted by sumrtym

I'm really not liking the lack of Season / Episode data. To me, that's a must have.

so then where did Sage get that data from? Can we subscribe to that service individually - create a small company - lol

[KeithAbbott]

Maybe it's time to get a SageTV representative on the Advisory Council or on the Board of the Schedules Direct organization. Any volunteers? Slugger?

[Slugger]

Originally Posted by kricker

Schedules Direct costs $20 a year is that right? Is that the kind of subscription we will need with them to use this plugin?

Yes, the $20/yr subscription is what would be needed.

Originally Posted by sumrtym

I'm really not liking the lack of Season / Episode data. To me, that's a must have.

Me neither, but it is what it is. The 'live' airing flag is very important to me as well. The season/series premiere/finale flags aren't critical, but I've come to know and love them. mc2xml does not provide this info nor does xmltv. I'm starting to think that Sage generated/pulled this info in from elsewhere or Tribune charges a massive premium for these details. However, I'm leaning towards the former because if it were available, but with a big premium slapped on it then would Sage ever have been in the position to pay the premium, but not Microsoft (source of mc2xml data, which doesn't carry these details)? I think Jeff pulled this info in from some other source to augment the Z2I data. If I'm right, maybe he'd share those details because I can pull it in and make use of it just as easily.

Originally Posted by tvmaster2

so then where did Sage get that data from? Can we subscribe to that service individually - create a small company - lol

See above. I think we need Jeff to share.

Originally Posted by KeithAbbott

Maybe it's time to get a SageTV representative on the Advisory Council or on the Board of the Schedules Direct organization. Any volunteers? Slugger?

That's a pass from me. "Advisory Council" sounds like a lot of red tape and discussions. I'll leave that to someone else and stick to the coding.

[kmp14]

Originally Posted by KeithAbbott

Maybe it's time to get a SageTV representative on the Advisory Council or on the Board of the Schedules Direct organization. Any volunteers? Slugger?

Doesn't slugger do enough for us already?

[nyplayer]

Slugger,

Is it possible to make your plugin compatible with mc2xml ?

[Slugger]

Originally Posted by nyplayer

Slugger,

Is it possible to make your plugin compatible with mc2xml ?

Not this one. This one will be Schedules Direct only because it doesn't require any additional tools/programs - this plugin talks directly with the SD server and requires no "middleman" programs/tools. This plugin can be installed and left alone.

Since mc2xml just generates xmltv output, you should be able to use the xmltv importer plugin with the output of mc2xml. I've not really studied the xmltv plugin, but since all non US and Canadian users have to use it for their data, I assume it works. To use mc2xml, just call it to generate the xml file you'd feed into this plugin instead of calling xmltv (I assume this is possible, I haven't studied how this plugin works at all).

When the need arises for EPG plugins for North America, I might be convinced to write another xmltv plugin allowing for a similar installation/setup for an xmltv EPG source, but, admittedly, it will take quite a bit of convincing since I'm only going to need this plugin for my setup.

[sdsean]

Slugger, does the roll back - uninstall of this plugin happen properly?
Just want to make sure, b/c I want to try this out in case the google thing goes south.
Also are the channel ids compatible? The reason I ask is that have some channels added to my lineup that are actually Canadian cable channels, but they have the all important game info missing in SageTV when using a DirecTV lineup from the U.S. So what I do is create new channels from a Canadian lineup using Zap2It, and the map them to the appropriate DirecTV channel.

(Of course maybe your plugin will just give me the guide data anyway. . .i never understood why SageTV doesn't have this yet BeyondTV did, and they get their data from the same place).

[Slugger]

Originally Posted by sdsean

Slugger, does the roll back - uninstall of this plugin happen properly?
Just want to make sure, b/c I want to try this out in case the google thing goes south.

This is not something I've tested yet. The install isn't exactly perfect so the uninstall will be tricky, I assume, as well. Since this plugin modifies the epg/epg_import_plugin property when installed, you must manually restart SageTV after installing this plugin before the core will start using the EPG plugin - there is no way for me to tell you it needs to be restarted. The only other option I have is to just force a restart, but that seems like a bad idea so I didn't do that.

Uninstalling is also a manual process, unfortunately. There is no hook within plugins to catch an uninstall request from the user so I don't actually know when a user uninstalls it. So if you uninstall it, you have to manually stop Sage and remove the entry in epg/epg_import_plugin in Sage.properties. Not ideal, but it's the best I can do.

This plugin, though functioning on my test server, is no where near beta quality so don't expect anything released anytime soon. Furthermore, I can't release it until SD gives me an exemption, which isn't likely to happen until Sage turns off its EPG service.

Also are the channel ids compatible? The reason I ask is that have some channels added to my lineup that are actually Canadian cable channels, but they have the all important game info missing in SageTV when using a DirecTV lineup from the U.S. So what I do is create new channels from a Canadian lineup using Zap2It, and the map them to the appropriate DirecTV channel.

(Of course maybe your plugin will just give me the guide data anyway. . .i never understood why SageTV doesn't have this yet BeyondTV did, and they get their data from the same place).

The ids are compatible (based on my very quick inspection), but, unlike the core EPG service, Schedules Direct does not allow arbitrary additions of extra channels to lineups. This plugin is only capable of accessing the data as setup in your SD account. So you select and configure your lineup(s) on the SD web site and whatever you configure there is what this plugin pulls in and uses. From what I can see, there is no way to add additional channels to a lineup on the SD site. You select a lineup (i.e. "DirecTV") and the SD site gives you the channels for that lineup. You can disable channels from your lineup, but that's it.

[KeithAbbott]

Originally Posted by Slugger

The ids are compatible (based on my very quick inspection), but, unlike the core EPG service, Schedules Direct does not allow arbitrary additions of extra channels to lineups. This plugin is only capable of accessing the data as setup in your SD account. So you select and configure your lineup(s) on the SD web site and whatever you configure there is what this plugin pulls in and uses. From what I can see, there is no way to add additional channels to a lineup on the SD site. You select a lineup (i.e. "DirecTV") and the SD site gives you the channels for that lineup. You can disable channels from your lineup, but that's it.

So if I use a combination of cable and OTA, and select my cable lineup at the SD site, any additional OTA channels that I receive that are not available from my cable provider will not get any guide data?