Finally! Fully integrated free Canadian Listings

[Lester Jacobs]

Calling all Canadians!

Zap2It has recently made available their Zap2It Data Direct service which supplies high quality guide data for the North American market. This guide data provides better content and is more reliable than the XMLTV feed that we (outside the USA) are currently limited to using.

I have built a fully integrated EPG plugin for SageTV using this Data Direct service. Since it is a fully integrated plugin it behaves just like the builtin Sage Guide service for the US market. There are no batch files to configure or any epgdata.xml files to download and move around. Once installed the plugin will keep your guide listings up to date with no intervention on your part.

Look for release of the plugin this weekend. Currently I'm putting the code through its final testing phase.

Stay tuned!

Cheers
Lester

UPDATE: The plugin is now ready. You can download it and get instructions on use at DigiCasa
Please post any issues or questions in this thread.

Latest version: Version 1.20, May 9, 2004

Support Forum In order to provide better support I've setup a forum at DigiCasa . Please visit there to log support requests.

Thanks
Lester

[Grungy Remnant]

Sweet! Looking forward to trying that puppy out.

[olyar15]

Wow, great news. Although I have been happy using the XMLTV, an integrated solution is certainly better. Thanks for writing this plugin.

[Lester Jacobs]

The plugin is now ready! Check the first post for details.

Cheers
Lester

[olyar15]

Got it working, and so far looks good. One thing I noticed is that the current hour has no data. I installed the plugin just past 9PM, so that hour has no data, but everything after that is fine. Just a warning to others that you should try to install this just before the hour is up, to minimize the lack of data. Other than that, it certainly looks promising. A big thanks to Lester for creating this plugin.

[mikbro]

Originally posted by Lester Jacobs
The plugin is now ready! Check the first post for details.

Cheers
Lester

This is super Lester!

I am running into a problem though - here is my log entries from Sage 2.0.13 ...

Sun 3/7 0:27:12.464 Parsing downloaded listings...
Sun 3/7 0:27:12.620 org.xml.sax.SAXParseException: The value of attribute "channel" must not contain the '<' character.
org.xml.sax.SAXException: Stopping after fatal error: The value of attribute "channel" must not contain the '<' character.
Sun 3/7 0:27:12.626 at org.apache.xerces.framework.XMLParser.reportError(XMLParser.java:1245)
Sun 3/7 0:27:12.640 at org.apache.xerces.framework.XMLDocumentScanner.reportFatalXMLError(XMLDocumentScanner.java:598)
Sun 3/7 0:27:12.641 at org.apache.xerces.framework.XMLDocumentScanner.scanAttValue(XMLDocumentScanner.java:528)
Sun 3/7 0:27:12.641 at org.apache.xerces.framework.XMLDocumentScanner.scanElement(XMLDocumentScanner.java:1769)
Sun 3/7 0:27:12.642 at org.apache.xerces.framework.XMLDocumentScanner$ContentDispatcher.dispatch(XMLDocumentScanner.java:1182)
Sun 3/7 0:27:12.642 at org.apache.xerces.framework.XMLDocumentScanner.parseSome(XMLDocumentScanner.java:381)
Sun 3/7 0:27:12.643 at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1098)
Sun 3/7 0:27:12.644 at zap2it.Zap2ItImportPlugin.updateGuide(Zap2ItImportPlugin.java:96)
Sun 3/7 0:27:12.644 at sage.l.b(Unknown Source)
Sun 3/7 0:27:12.645 at sage.o.char(Unknown Source)
Sun 3/7 0:27:12.645 at sage.m.aF(Unknown Source)
Sun 3/7 0:27:12.649 at sage.l.run(Unknown Source)
Sun 3/7 0:27:12.649 at java.lang.Thread.run(Unknown Source)
Sun 3/7 0:27:12.650 ERROR Updating EPG Source Zap2It Data Direct Lineup
Sun 3/7 0:27:12.651 ErrorText:expand called on Zap2It Data Direct Lineup at Sun 3/7 0:27:14.707 expandedUntil=Wed 12/31 19:00:00.000 confirmedUntil=Wed 12/31 19:00:00.000
EPG Expanding Zap2It Data Direct Lineup at Sun 3/7 0:27:14.709


Looks like there is something in the data that is invalid... could you possibly make the source available so I can look into the problem?

Thanks!

[jmiddleton]

Quick and easy install - the biggest problem I has was forgetting to fill in a survey question on the Zap2It site! As Olyar15 pointed out there is no data for programs currently playing. Hopefully this is an artifact of the initial install and won't affect updates. Otherwise this looks like a supurb tool that should attract a lot of Canadians to SageTV. Great work and thanks for making it available to the community.

[Lester Jacobs]

Originally posted by olyar15
Got it working, and so far looks good. One thing I noticed is that the current hour has no data. I installed the plugin just past 9PM, so that hour has no data, but everything after that is fine. Just a warning to others that you should try to install this just before the hour is up, to minimize the lack of data. Other than that, it certainly looks promising. A big thanks to Lester for creating this plugin.

Good catch! I've updated the plugin so that it now grabs data starting one second after midnight instead of the current time. This should eliminate the problem.

Please download the plugin again and see if the problem persists.

Be careful to save your zap2itconfig.xml file though as unzipping the plugin will overwrite this file and you'll have to re-enter your username and password.

Cheers
Lester

[Lester Jacobs]

Originally posted by mikbro
This is super Lester!

I am running into a problem though - here is my log entries from Sage 2.0.13 ...

Sun 3/7 0:27:12.464 Parsing downloaded listings...
Sun 3/7 0:27:12.620 org.xml.sax.SAXParseException: The value of attribute "channel" must not contain the '<' character...

Hi Mike,

Can you check the file zap2itdata.xml which should be in your SageTV directory. This contains all of the downloaded data from Zap2It Data Direct. Look for entries in this file resembling the following:

<map station='20783' channel='26'/>
<map station='15024' channel='27'/>
<map station='11953' channel='28'/>

From your error message, it looks like there's somethig wrong with the channel attributes. Can you check all of your channel attributes to see if any of them are non-numerical or have any strange characters?

Cheers
Lester

[mikbro]

I took a look at the zap2it error log, and it showed a "unexpected error" when attempting to get a lineup (I have 2 lineups in my profile currently).

So it does not look like a problem with the plugin, just tribune.

Thanks!

[pawn]

Hi Lester:

I'm really excited about this, and thank you for your efforts...

...but, there doesn't seem to be a lot of functionality for multiple tuner / source setups.

I have two tuners with a cable source and one with a satellite source. I have setup both sources with zap2it with no problems. But there's no way to discriminate which data source to use in the Setup Wizard.

This didn't look like a problem at first, since Sage 2.0.13 lets you select unqiue channel lineups for each tuner: but that doesn't seem to work. I'm not sure if it's a Sage bug or a problem with the plugin, but when I select/deselect a channel for one tuner, it does it for all tuners.

(or maybe I'm out to lunch...but I don't think so)

Thanks again.

KJ

[Lester Jacobs]

Originally posted by pawn
Hi Lester:

I'm really excited about this, and thank you for your efforts...

...but, there doesn't seem to be a lot of functionality for multiple tuner / source setups...

Ok, I took a lok at how Zap2It handles lineups. Since I only use one lineup, I'd never tested the multi-lineup situation. However, I've now setup 2 lineups for myself. Seems they dump the information for all lineups into one file, there's no way to tell them just to download the info for one lineup. Therefore I have to do a little bit of re-architecting of the plugin to handle the multi-lineup situation.

I'll work on this and post the updated plugin. It'll be done sometime this week (my "real" job demands a chunk of my time as well).

Cheers
Lester

[pawn]

Originally posted by Lester Jacobs
Ok, I took a lok at how Zap2It handles lineups. Since I only use one lineup, I'd never tested the multi-lineup situation. However, I've now setup 2 lineups for myself. Seems they dump the information for all lineups into one file, there's no way to tell them just to download the info for one lineup. Therefore I have to do a little bit of re-architecting of the plugin to handle the multi-lineup situation.

I'll work on this and post the updated plugin. It'll be done sometime this week (my "real" job demands a chunk of my time as well).

Cheers
Lester

Let me know if I can help in any way. No rush.

I wish I knew a little more about how Sage handles plugins. It seems to me your existing design is workable as is, but that the tuner specific channel setup in Sage 2.0.XX isn't working as expected. I.e., if I could just isolate channels above 200 to the sat tuner only, I'd be fine.

But for whatever reason, even though Sage lets you pick channels specifically for each tuner, changing one lineup changes them all. The question is, is that a flaw in Sage or a flaw in your plugin?

Thanks again, this is great. I noticed a big difference in the data during my brief trial.

[davey_fl]

Hi Lester,

Just wanted to let you know that the plug-in is fantastic! Very easy to install and the update speed it awesome. My xmltv was working fine but sometimes would take 4 hours to complete. I loaded 11 days of EPG data in just under 40 seconds, AMAZING! Thanks very much for sharing with us!

Davey

[Lester Jacobs]

Hi All,

Multiple lineup support is now included in the plugin. The plugin will now allow you access to all of the lineups that you have configured at the Zap2It Data Direct website. Please download and install the plugin as indicated in the first post in this thread.

Please let me know how this works out for you. Although I have tested it, I do not use multiple lineups in my setup and therefore I'd like some feedback from those who do have multiple lineups.

Regards
Lester

[pawn]

Hey L:

Again, nice work. The multiple source engine works now. A few things though:

- When entering the ZIP code for the first source entered, Sage freezes for almost a minute (no warning or heads up that something's happening). It gives that feeling that Sage has crashed altogether.

- The first few channels tuned to after setting up my three sources all tuned a sat channel (not what I was trying to tune to). After playing a bit, everything worked well, ...except:

- Most importantly: all shows in the guide are shown as First Run. Obviously this is a big problem, although, maybe not a problem with your plugin. But in all honesty, if there's no first run / rerun data in this new zap2it data, I personally would go back to XMLTV, where at least I can parse the data and add this info to the shows I watch. I find this info critical to how stable Sage is.

[Lester Jacobs]

Originally posted by pawn
Hey L:

- When entering the ZIP code for the first source entered, Sage freezes for almost a minute (no warning or heads up that something's happening). It gives that feeling that Sage has crashed altogether.

Yes, this is an unfortunate side effect of how Zap2It provides listings. The plugin has to download all of the data, not just the lineup information, in order to show lineups. I did highlight the possible long pause on the instruction page though.

- Most importantly: all shows in the guide are shown as First Run. Obviously this is a big problem, although, maybe not a problem with your plugin. But in all honesty, if there's no first run / rerun data in this new zap2it data, I personally would go back to XMLTV, where at least I can parse the data and add this info to the shows I watch. I find this info critical to how stable Sage is.

The First Run problem is in the data and therefore the plugin can't do anything about it at this stage. However, there are two points that might help here

1) If a show is being aired at multiple times on multiple channels, the new plugin assigns a single internal id to this show. Therefore SageTV "knows" it is the same show even if it appears as a First Run in all airings. Therefore SageTV should not try to record the same show twice, etc. The old xmltv plugin assigns a unique id to each airing of the show, resulting in Sage thinking that each airing of the same show actually represents a new show. This might cause it to record the same show multiple times.

2) I am looking at providing a mechanism for users to specify first run/rerun status as well the ability to change other aspects of show information. This would allow users to specify known first runs and reruns for shows that they are interested in.

Cheers
Lester

[pawn]

The First Run problem is in the data and therefore the plugin can't do anything about it at this stage.


(sigh) I was hoping you were going to say I was doing something wrong.

In all honesty, without this, the data isn't much better than the scraped data. I could care less about listing actors, etc. I care about avoiding confilcts, because, in my opinion, that's when Sage (at least in the 1.X.XX versions) starts acting flaky.

[pawn]

If a show is being aired at multiple times on multiple channels, the new plugin assigns a single internal id to this show.


I'm glad you mentioned this, because I now understand what is happeneing when I select a show not on the sat channel range, and it tunes to a sat channel: when the same episode is on one of my cable tuners as well as the sat tuner, if I tune to that show on a channel that is only in the cable channle range, the plugin will sometimes tune to the sat listing instead.

Not sure if that's clear.

[pawn]

More feedback:

Came home today and had no guide data using your new plugin (had been running continuously for about 5 days).