New XMLTV Import Tool

[StephaneM]

Originally Posted by Madcoder

So if someone has donated for the BeyondTV version, but these days uses Sage, can they still obtain the new version for Sage, or is there a new donation required?

You can in theory use the BTV XMLTV Importer with the Java classes you can found in the install folder of the SageTV XMLTV Importer, and points the Java classes to the location of file produced by the BTV Importer.

For now the XML files produced by both Importer are compatibles with some restrictions (for instance : you have to have the upcoming version of the BTV Importer and be running a BTV 3.5 version or newer) and I could not guarantee you that this will be always the case.

So you'd better should make a new donation, because this way you'll receive e-mail notifications of new update of the Importer for the PVR you use.

Regards,
Stéphane.

[johnnytyler]

I've tried the importer, and I can't seem to get it to work

1) I downloaed the info from zapit one days epg and set up a Dish.xml file
2) I ran the importer and configured the channels list, deleted the channels I was not using from the channel editor and added numbers
3) after running the importer, it outputed 4 files to 3rd party guid data folder, episodes, lineup, linup info by index, and programs. However all 3 xml files are only 1 kb except for the linup. I dont think that is right.
4) Sagetv loads the channel info just fine, however when I go to guide there is NO DATA in every field.

I've checked the original xml and it does contain the program info from zapit

[StephaneM]

Hello,

Originally Posted by johnnytyler

1) I downloaed the info from zapit one days epg and set up a Dish.xml file
2) I ran the importer and configured the channels list, deleted the channels I was not using from the channel editor and added numbers it does contain the program info from zapit

From what I read here, I guess that :

* You don't have set properly the "Time Offset" in the Importer, this is especially critical if you download only one day of programs : the Importer only converts the program that you can view in you local time zone. So if the "time offset" is wrong, all the show in your XMLTV file can be "out of air". (Note: The Importer doesn't trust the time zone found in the XMLTV file, that's why he ask you to specify your time zone properly, for instance if in the XMLTV file you have "20060324145000 +1000" as program start date, you'll have to specify -10 as the time offset : basically the "time offset" property is added to "20060324145000" to retrieve the UTC time).

* You don't have edited all your channels properly. I don't know how you can delete unused channels in the channel editor : you are not supposed to be able to delete anything in the "Channels" tab (you can however add and delete channels in the "Additionnal channels" tab).

You can review your Windows Event Viewer, the "SageTV Utilities" log will surely show some warnings like "No programs imported for channel xx".

Regards,
Stéphane.

[owilsky]

1. One Bug: In Windows Scheduler the task's name is "BTV XMLTV Import Task", guess you should change that ;-)

2. One Question: How much do you have to donate to get the full version?

Oliver

[StephaneM]

Hello Oliver,

Originally Posted by owilsky

1. One Bug: In Windows Scheduler the task's name is "BTV XMLTV Import Task", guess you should change that ;-)

I had to forget something

2. One Question: How much do you have to donate to get the full version?

There is no defined amount, this is a donation, so you really donate what you want / can / think it deserves. Whatever the amount of your donation you receive the same support (the newsletter subscription, usually I release updated version of the software once a month) and software.

Regards,
Stéphane.

[owilsky]

Thanks! I just figured to set it up. But I had to delete several xerces.zip's to make it work.

One more question, I have several warnings in the xmltv.log:

26-04-2006 18:14:06,663 Reading Episodes File : Episodes0.xml
26-04-2006 18:14:07,304 Adding airing to SageTV DB
26-04-2006 18:14:07,460 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,507 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,601 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,663 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,710 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,741 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:10,913 Failed to add airing for 'SH0020310001' to DB
26-04-2006 18:14:11,069 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,194 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,241 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,288 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,351 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,382 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,476 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:15,429 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,491 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,523 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,569 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,648 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,741 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,835 Failed to add airing for 'SH0015860001' to DB
26-04-2006 18:14:16,132 Failed to add airing for 'SH0015860001' to DB
26-04-2006 18:14:21,069 13555 airings added
26-04-2006 18:14:21,069 End Update

Do I have to worry about that? Seems to work fine though... now I have to check if reruns are recognized as such ;-)

[StephaneM]

Originally Posted by owilsky

Thanks! I just figured to set it up. But I had to delete several xerces.zip's to make it work.

Yes, I only rename xerces.jar to xerces.jar.old if found in the JARs directory of SageTV, I will add xerces.zip to the process (however I'm not looking in the Java directory).

One more question, I have several warnings in the xmltv.log .../... Do I have to worry about that?

This error means SageTV has refused to add the airing into the DB for the episode specified.

To see if it is really a problem, you can look in the "Episodes0.xml" and in the "Programs0.xml" too see what show it concerns.

First : open the "Programs0.xml" (in the SageTV\3rdPartyGuideData folder) in Internet Explorer, wait until the file is completely loaded (this could take some time) and then search for "SH0003050001", and you'll find what the show is.

Next: open the "Episodes0.xml" in Internet Explorer and again search for "SH0003050001", you'll then have all airings for this particular show, take a note of all airings date time (they are in UTC time in this file), then you can compare it with your EPG in SageTV to see if there is something missing.

Unfortunately, all I know is SageTV has returned "False" as a result of the call to add the airing in the database. Basically the information passed to SageTV is : the EPGID, the Channel ID, the start time and the duration.

The duration is computed from the show start time and stop time, it is possible that the duration is wrong or something like that. It seems to happen only for 5 shows, so these five must have something similar that prevent them to be added.

Regards,
Stéphane.

[owilsky]

Originally Posted by StephaneM

First : open the "Programs0.xml" (in the SageTV\3rdPartyGuideData folder) in Internet Explorer, wait until the file is completely loaded (this could take some time) and then search for "SH0003050001", and you'll find what the show is.

Hmmm... none of these IDs are in the Programs0.xml, only in the Episodes0.xml.

[StephaneM]

Originally Posted by owilsky

Hmmm... none of these IDs are in the Programs0.xml, only in the Episodes0.xml.

Okay... could you send me the content of your 3rdPartyGuideData folder zipped to lmgestion@hotmail.com

I'll take a look at those files and let you know.

[DIBU]

i cant rename or delete xerces, some other software on my server will use this

Originally Posted by owilsky

Thanks! I just figured to set it up. But I had to delete several xerces.zip's to make it work.

One more question, I have several warnings in the xmltv.log:

26-04-2006 18:14:06,663 Reading Episodes File : Episodes0.xml
26-04-2006 18:14:07,304 Adding airing to SageTV DB
26-04-2006 18:14:07,460 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,507 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,601 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,663 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,710 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:07,741 Failed to add airing for 'SH0003050001' to DB
26-04-2006 18:14:10,913 Failed to add airing for 'SH0020310001' to DB
26-04-2006 18:14:11,069 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,194 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,241 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,288 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,351 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,382 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:11,476 Failed to add airing for 'SH0024100001' to DB
26-04-2006 18:14:15,429 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,491 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,523 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,569 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,648 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,741 Failed to add airing for 'SH0028270001' to DB
26-04-2006 18:14:15,835 Failed to add airing for 'SH0015860001' to DB
26-04-2006 18:14:16,132 Failed to add airing for 'SH0015860001' to DB
26-04-2006 18:14:21,069 13555 airings added
26-04-2006 18:14:21,069 End Update

Do I have to worry about that? Seems to work fine though... now I have to check if reruns are recognized as such ;-)

[StephaneM]

Originally Posted by DIBU

i cant rename or delete xerces, some other software on my server will use this

Normally, you should be able to put xerces.jar / xercer.zip in a local JAR directory relative to the application.

The prerequisite is that it should not be in the Java directory and in the SageTV JARs directory.

I was having the exact opposite problem as you : some application (ie : WebScheduler) was not working at all with xerces.jar in the Java directory... so that's why I choose not to use it.

Note: The error listed in the previous message form Oliver, was because of an error in the XMLTV file : the programs not imported had an incorrect start / stop time.

Note (bis) : I'm currently working on providing better support for Zap2It XMLTV file (specifically, using the dd_progid EPGID given by Zap2It and even reusing it when you use other XMLTV files with the Zap2It file and making some accomodation for non standard use of XMLTV DTD).

Regards,
Stéphane.

[rsagetv99]

Originally Posted by StephaneM

Note (bis) : I'm currently working on providing better support for Zap2It XMLTV file (specifically, using the dd_progid EPGID given by Zap2It and even reusing it when you use other XMLTV files with the Zap2It file and making some accomodation for non standard use of XMLTV DTD).

Great idea, I was kind of thinking along the same lines. Not sure what you are planning, but depending on how complicated it is, it would be nice if when you are using a non-Zap2it file along with a Zap2it file then the importer would get the Zap2it dd_progid EPGID from your internal database and assign it to the same episode from the non-Zap2it file (assuming that no EPGID was provided). Of course this would require the importer to read the Zap2it files first, before an EPGID gets generated for the other file.

[DIBU]

Originally Posted by StephaneM

Normally, you should be able to put xerces.jar / xercer.zip in a local JAR directory relative to the application.

The prerequisite is that it should not be in the Java directory and in the SageTV JARs directory.

I was having the exact opposite problem as you : some application (ie : WebScheduler) was not working at all with xerces.jar in the Java directory... so that's why I choose not to use it.

Note: The error listed in the previous message form Oliver, was because of an error in the XMLTV file : the programs not imported had an incorrect start / stop time.

Note (bis) : I'm currently working on providing better support for Zap2It XMLTV file (specifically, using the dd_progid EPGID given by Zap2It and even reusing it when you use other XMLTV files with the Zap2It file and making some accomodation for non standard use of XMLTV DTD).

Regards,
Stéphane.

well done i have found the other app, but now i have a prob...on my dbox (well i live in germany so i have no zap) on the xmltv input i have no channels..

[StephaneM]

Originally Posted by DIBU

on my dbox (well i live in germany so i have no zap) on the xmltv input i have no channels..

Do you mean that:

* when you are using the Channel Editor of the Importer it complains it cannot found any channels

* or when you are choosing the XMLTV option in SageTV then it returns no channels

In both cases please check your Windows Event viewer Log you'll find here all warnings and errors encountered during the transformation process of your XMLTV file by the Importer.

Then check the c:\Program Files\SageTV\SageTV\xmltv.log file, this one will contain log concerning the load of your data by SageTV using the Java classes of the Importer.

Usually what is going wrong can be :

* You have not correctly set up your lineup and channels in the Channel Editor (in this case it will log in the Event Viewer that no channels / lineup were defined)

* You have not set the appropriate time offset in the Importer (the Importer doesn't care at all of the time zone found in the XMLTV file : because some XMLTV file have one, some other don't, so it's more reliable to have this setting in the Importer itself). If the time offset is wrong and you have a few data, then the Importer may think that all data is out of air (and in this case it will log this in the Event Viewer)

* The Importer was not set up correctly. A good setup include :

- Copying Java Classes in the C:\Program Files\SageTV\SageTV\xmltv folder (XMLTVImportPlugin.class, Channel.class,Episode.class,Program.class)
- Creating a xmltv.properties file in C:\Program Files\SageTV\SageTV folder (the file contain just one parameter pointing to the C:\Program Files\SageTV\SageTV\3rdPartyGuideData folder)
- Adding the plugin in the C:\Program Files\SageTV\SageTV\Sage.properties file (adding this line : epg/epg_import_plugin=xmltv.XMLTVImportPlugin)

When running the Importer should produce XML files into the C:\Program Files\SageTV\SageTV\3rdPartyGuideData folder)

Let me know.

Regards,
Stéphane.

[StephaneM]

Originally Posted by rsagetv99

it would be nice if when you are using a non-Zap2it file along with a Zap2it file then the importer would get the Zap2it dd_progid EPGID from your internal database and assign it to the same episode from the non-Zap2it file (assuming that no EPGID was provided).

Yes this is what it should be doing in the upcoming release.

Regards,
Stéphane.

[DIBU]

Originally Posted by StephaneM

Do you mean that:

* when you are using the Channel Editor of the Importer it complains it cannot found any channels

* or when you are choosing the XMLTV option in SageTV then it returns no channels

In both cases please check your Windows Event viewer Log you'll find here all warnings and errors encountered during the transformation process of your XMLTV file by the Importer.

Then check the c:\Program Files\SageTV\SageTV\xmltv.log file, this one will contain log concerning the load of your data by SageTV using the Java classes of the Importer.

Usually what is going wrong can be :

* You have not correctly set up your lineup and channels in the Channel Editor (in this case it will log in the Event Viewer that no channels / lineup were defined)

* You have not set the appropriate time offset in the Importer (the Importer doesn't care at all of the time zone found in the XMLTV file : because some XMLTV file have one, some other don't, so it's more reliable to have this setting in the Importer itself). If the time offset is wrong and you have a few data, then the Importer may think that all data is out of air (and in this case it will log this in the Event Viewer)

* The Importer was not set up correctly. A good setup include :

- Copying Java Classes in the C:\Program Files\SageTV\SageTV\xmltv folder (XMLTVImportPlugin.class, Channel.class,Episode.class,Program.class)
- Creating a xmltv.properties file in C:\Program Files\SageTV\SageTV folder (the file contain just one parameter pointing to the C:\Program Files\SageTV\SageTV\3rdPartyGuideData folder)
- Adding the plugin in the C:\Program Files\SageTV\SageTV\Sage.properties file (adding this line : epg/epg_import_plugin=xmltv.XMLTVImportPlugin)

When running the Importer should produce XML files into the C:\Program Files\SageTV\SageTV\3rdPartyGuideData folder)

Let me know.

Regards,
Stéphane.

i found no error on logs..when i go to chanel setup..i see no channels ..the dbox becomes a signal (can i see in the console)

xmltv is on the menu, so that i can use xmltv as input..but no channels comes up

[StephaneM]

Originally Posted by DIBU

i found no error on logs..when i go to chanel setup..i see no channels ..the dbox becomes a signal (can i see in the console)

At least do you see something in the Event Viewer under "SageTV Utilities", it should show you the begining and ending of the transformation process in the logs...

xmltv is on the menu, so that i can use xmltv as input..but no channels comes up

Do you see the lineups you created in the Importer Channel Editor? What do you see in the C:\Program Files\SageTV\SageTV\xmltv.log?

Do you have XML files in the C:\Program Files\SageTV\SageTV\3rdPartyGuideData?

The best thing you can do for now, is stopping completely SageTV (client and service), then uninstall the Importer, install it again (you will not loose your configuration settings). Then start SageTV and verify that all is setup correctly (Java classes, xmltv.properties, and Sage.properties).

Then start the Importer to prepare XMLTV data for SageTV (review your channels in the channel editor). When the Import has finished, go into SageTV and configure your tuners to use XMLTV data.

Regards,
Stéphane.

[svalmorri]

Hi,

question that applies to your importer but I guess it's SageTV wide: when in Channel Editor you specify the channel number for your lineup(s), that No. is the RF number, isn't it?

Is there any way in the importer or in SageTV to virtualize the channels to appear with different and more confortable numbering? Example: RAI1 is channel (RF) no. 63, I would like it to show as no. 1 in the lineup and in the EPG.

Is it possible? I have found some information in the former posts but it's a bit confusing.

Thanks

[rsagetv99]

Originally Posted by StephaneM

Yes this is what it should be doing in the upcoming release.

Regards,
Stéphane.

Don't forget to remove the "." that is before the last 4 digits of the ID, it appears that this is what the internal EPG does so unless you do it to then the IDs won't match.

[StephaneM]

Originally Posted by svalmorri

when in Channel Editor you specify the channel number for your lineup(s), that No. is the RF number, isn't it?

Yes it is the RF number for analog sources (Antenna / Cable) or the logical channel number for other sources (ex: SBDARecorder, MyTheaterPlugin, Native DVB Support).

Is there any way in the importer or in SageTV to virtualize the channels to appear with different and more confortable numbering? Example: RAI1 is channel (RF) no. 63, I would like it to show as no. 1 in the lineup and in the EPG.

No this is not possible. If you remap the channel in SageTV to use another channel number, it will use this number to tune the channel.

You may have another options but I'm not aware of them (I began to use SageTV as my main PVR only three weeks ago). I guess it is as with all other PVR : you'll have to edit the DirectShow Frequencies to trick SageTV (the RF number 1, will indeed tune to RF number 63), there is some utilities you can find to change these frequencies, I never used them myself, because I don't bother at all that the channel numbers be the RF number.

It's a cultural thing: in USA TV users does know the RF number and only this one (they even base the Digital TV numbering system on this number even though the real RF is another one), whereas in Europe we don't even know on what RF a channel is on...

Regards,
Stéphane.