New XMLTV Import Tool

[alon24]

do you use java 1.5 and up?
Try to installe sage, run first time, close it (and service too)
install XMLIMPORTER 2.5 (it will install the files also in sage)
run the importer (so that it will create the appropriate files)
run sage, delete the source (if u added one before)
create a source, and now try to see if it works (should work now)

[StephaneM]

Hello,

Originally Posted by Sterf

I can't get it to work in Vista.../...I select 'use xmltv data', then cable, use postal 0000 like I did in XP, and it won't show me the XMLTV lineup to download.

You'll have to check that :

* the sage.properties file does include the xmltv plugin declaration
* the xmltv.properties file does point to the folder where the guide data are produced by the Importer
* you are using Java runtime 1.5 or newer.
* you can also review the sagetv.plugin.log and the Windows Event Viewer to see if any errors are listed

You can review the FAQ for the files location (Guide data and plugin log), for Vista this is :

C:\ProgramData\LM Gestion\SageTV XMLTV Importer\GuideData

Also you'll have to search for the sage.properties file : its location depends on UAC / Account used for elevation / Account used for SageTV service...

The xmltv.properties file should be in C:\Program Files\SageTV\SageTV folder

Regards,
Stéphane.

[Sterf]

Thanks for the help. Turned out I just had to update java. And I turned that irritating UAC off, nothing worked with it on. And to be completely offtopic, apparantly since I have Vista all the physical numbers for my channels changed. Which is strange.

[StephaneM]

Originally Posted by Sterf

And I turned that irritating UAC off, nothing worked with it on.

My personal advice would be to let turn it on on a regular PC (that is one you use for work, reading e-mails, surfing the Internet) and eventually turn it off on a dedicated HTPC (allthough I do find that currently Vista as an HTPC is not an easy choice namely because hardware acceleration seems to not be working the same as in XP, well I can't say for SageTV but it's definitely the case with Media Player / Media Center)

And to be completely offtopic, apparantly since I have Vista all the physical numbers for my channels changed. Which is strange.

I don't know, at least for me in SageTV the frequencies are exactly the same for digital channels (DVB-T), I can't say if the directshow frequency table for analog channels for aerial and cable are the same as in XP but I don't see why they would change.

Regards,
Stéphane.

[LazyGun]

Has anyone seen this error message in the sage.plugin.log?

02-05-2007 22:24:01,875 Start EPG update for lineup n°1275967375773
02-05-2007 22:24:01,875 Reading Channels File
02-05-2007 22:24:01,875 Creating channels array (17)
02-05-2007 22:24:01,875 Unexpected error for 'updateGuide'
java.lang.NumberFormatException: For input string: "1275967375773"
	at java.lang.NumberFormatException.forInputString(Unknown Source)
	at java.lang.Integer.parseInt(Unknown Source)
	at java.lang.Integer.parseInt(Unknown Source)
	at xmltv.XMLTVImportPlugin.updateGuideEx(XMLTVImportPlugin.java:248)
	at xmltv.XMLTVImportPlugin.updateGuide(XMLTVImportPlugin.java:87)
	at sage.z.q(Unknown Source)
	at sage.aw.void(Unknown Source)
	at sage.a3.cJ(Unknown Source)
	at sage.z.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

I had a look through the thread and couldn't see anything (tried searching as well), so apologies if this has been answered already.

The number it is complaining about corresponds to one of the epg/channel_lineups/ entries in the sage.properties file.
I have jre 1.5.11 installed I believe, and v2.5 of the importer.

Thanks

[StephaneM]

Hello,

02-05-2007 22:24:01,875 Start EPG update for lineup n°1275967375773

This lineup number is not a usual lineup number for the XMLTV Importer (usually it is a very small integer 1, 2 or 3...)

So I guess that you were using another XMLTV Plugin and the lineup number used by this plugin is still in the sage.properties file and somehow SageTV is asking my plugin to load this lineup.

Either you can ignore the error completely if you can update your SageTV guide data normally (because obviously even if the error was caught the Importer will not load any data for this lineup anyway). Or if you are not able to load any guide data, you'll have to get rid of this extraneaous lineup reference.

Regards,
Stéphane.

[LazyGun]

Hmm, ok.
I have 2 tuners, and 2 lineups configured. One is the regular EPG, and one is using XmlTV.

I think I'll try uninstalling the plugin and then re-installing it, to see what kind of epg numbers I get in my sage.properties file again.

Thanks

[StephaneM]

Originally Posted by LazyGun

Hmm, ok.
I have 2 tuners, and 2 lineups configured. One is the regular EPG, and one is using XmlTV.

If I remember correctly you have to tell SageTV that you will use both XMLTV and native EPG. See this thread.

Anyway I would advise you to not do this: the EPGID of the native EPG and XMLTV will not be the same, so you'll have a hard time handling duplicate recordings.

I think I'll try uninstalling the plugin and then re-installing it, to see what kind of epg numbers I get in my sage.properties file again.

re-installing the XMLTV plugin will not resolve anything by itself.

Regards,
Stéphane.

[LazyGun]

Originally Posted by StephaneM

If I remember correctly you have to tell SageTV that you will use both XMLTV and native EPG. See this thread.

Anyway I would advise you to not do this: the EPGID of the native EPG and XMLTV will not be the same, so you'll have a hard time handling duplicate recordings.

re-installing the XMLTV plugin will not resolve anything by itself.

Regards,
Stéphane.

They are actually two different sets of channels on each lineup.
I'm messing with the Firewire recording of HD stuff, so my regular tuner has all my non HD channels on it, and the firewire tuner is using XmlTV to load the channel guides for the HD channels only.
So I should be ok with the EPGID I think.
I'll check out that thread though.

Thanks

[StephaneM]

Originally Posted by LazyGun

I'm messing with the Firewire recording of HD stuff, so my regular tuner has all my non HD channels on it, and the firewire tuner is using XmlTV to load the channel guides for the HD channels only.
So I should be ok with the EPGID I think.

Maybe, maybe not it's totally up to the way you'll want to record shows. On HD and Non HD channels some shows will be the same (except one show is aired in HD and the other in SD) so they should share the same EPGID, this way SageTV will be able to choose to record in HD or SD depending on the tuner use and personal preferences.

Regards,
Stéphane.

[Gabriel]

Stéphane, it is great to see how you keep on making this tool better and better!
I have a question on the latest update, v. 2.6. In my lineup I have channels with full episode tittles and description, and others with only the program name. For this reason I had to change the setting for "Unique program detection" from the default "ep. tittle or program description" to "ep. tittle or airing date". This way at least I have a different EPG ID for each day, but I still need to force manual recording of double features on the same day, for example. Your latest mod of changing the prefix to SH for these shows is a great improvement, thank you.
Now for my question: Should I change back the detection to "ep. tittle or program description" after I update?

[StephaneM]

Hello,

Originally Posted by Gabriel

Now for my question: Should I change back the detection to "ep. tittle or program description" after I update?

Yes, you should use the "ep. title or description" method. The "ep. title or airing date" is really for extreme case and with the recent changes doesn't serve any purposes anyway (I'll eventually remove the detection method choices and make the "ep. title or description" the default behavior).

Regards,
Stéphane.

[alon24]

HI,
Thanks for the updates in 2.6 I would try to use them tonight.

3 more requests if I may:

1. In the rule section, I would like to spcify a rule, but not have it stick to a specific show, but to a part of it only, and so:
not "prison break 2"
but: "prison break" - which will be true fo r "prison break 1", "prison break 2" ...

2. I sometimes see errors in specific channels, so to minimize the time it takes me to download the data, I trim the channels list to get from my source, to 3 or 4.

I tried it today with 2.6 and what it seemd to do, was change the channnels listed to that subset of channels (4 versus 30), loosing the channel physcal number (and probebly other data in the process).

Can you fix it, so that it does not loose sight of the full channels list, if I give it only a small list in the epg data xml?

3. I think I saw that the xmlimporter updates data about shows that are recording right now, and I think thats a problem.
I'll explain: Now is 19:02, I am recording a show, now my source has changed the epg, and I retrieved it.
The 19:00 ahow (that I am recording, has changed, weather if its a longer time to end at, or an earlier start time.
It does not matter, what I think i saw happening, is that the show would stop recording because of the update show information on it, and then resume when thats done.
I think that is a mistake to update a recording show, if its recording, dont break it.

WHat do you think?

[StephaneM]

Originally Posted by alon24

1. In the rule section, I would like to spcify a rule, but not have it stick to a specific show, but to a part of it only

This is not possible, the match is not made on title but on a program hash for spee reasons.

2. I sometimes see errors in specific channels, so to minimize the time it takes me to download the data, I trim the channels list to get from my source, to 3 or 4.../..Can you fix it, so that it does not loose sight of the full channels list, if I give it only a small list in the epg data xml?

Anytime you open the channels editor it will synchronise with your XMLTV files. If you want to cut down download time and be able to open channels editor without loosing your channels you can try to download one XMLTV file per channel set.

3. I think I saw that the xmlimporter updates data about shows that are recording right now, and I think thats a problem.

This is a SageTV issue if you have problems with recordings during a guide update. Anyway the Importer cannot know if a show is about to be recorded or not.

Regards,
Stéphane.

[alon24]

This is not possible, the match is not made on title but on a program hash for spee reasons.

i see you point about the speed, but I am just suggesting that it could be a better way, if i am willing to give up the speed for efficieny, and making it stick the way I want it, I mean I am willing to sucrifice it, and I am sure others will too. You could make it so there is a checkbox for partial or exact.
I am not complaining, I really appreciate all the hard work you are doing.

Anytime you open the channels editor it will synchronise with your XMLTV files. If you want to cut down download time and be able to open channels editor without loosing your channels you can try to download one XMLTV file per channel set.

I do not understand, I download one file per channel set, I only have 1 set. Do you mean, create 30 sets, and download 30 unique files?
and then upload them one at a time, or something like that? because I do not know in advance what will go wrong.

This is a SageTV issue if you have problems with recordings during a guide update. Anyway the Importer cannot know if a show is about to be recorded or not.

Actualy in another thread for a different xmlimporter, it was suggested (and imoplemented) that the importer will not update programs that start before the actuall time, so in essance if the program is currently recording it will not write over it.

[StephaneM]

Originally Posted by alon24

i see you point about the speed, but I am just suggesting that it could be a better way

It's really not that simple : the EPGID creation doesn't deal with the actual title of shows but only with hash . It's a design decision, the Importer is really doing a great number of lookups and it is needed.

I do not understand, I download one file per channel set, I only have 1 set. Do you mean, create 30 sets, and download 30 unique files?

The Importer can import more that one XMLTV files at once (and it see them as only one global source), so if you are downloading partial XMLTV files for better download speed : you can very well do something like this : monday download channels 1 to 10 in one XMLTV file, tuesday download channels 11 to 20 in one XMLTV file. The Importer will load both XMLTV files monday and tuesday. This way : it will see all your 20 channels.

Actualy in another thread for a different xmlimporter, it was suggested (and imoplemented) that the importer will not update programs that start before the actuall time, so in essance if the program is currently recording it will not write over it.

I'm not aware of any issues regarding update during recordings, but will see if there are and what they are. If thoses are minors issues : the Importer will still keep updating all non out of air shows (Nevertheless my previous statement is still valid : the Importer has no ways to know if a show is recording or not).

Regards,
Stéphane.

[ekiwi]

Yes, you should use the "ep. title or description" method. The "ep. title or airing date" is really for extreme case and with the recent changes doesn't serve any purposes anyway (I'll eventually remove the detection method choices and make the "ep. title or description" the default behavior).

ok i am a little confused.

i have many shows in my xmltv file that have

No episode name or id
and the same description for every show

eg <title>top of the hits</title><desc>Todays top hits</desc>


so as suggested above i changed to "ep. title or description"

and yes the shows get a ID similar to SHxxxx00002 (xxx is different for each title) the problem is all shows with the same title all get the same id so will only get recorded once if i mark one of them as watched they all get marked as watched.

i have removed all video sources and the lookup file still did not get a different id for each occurrence


if I use "ep. title or airing date" they get SHxxxxxyyyyy where yyyy increments for each day

am i missing something using rules or categories probably will not change anything as they are already marked as "SH" not as "EP"

so at the moment please do not remove the ability to choose the detection method

[StephaneM]

Originally Posted by ekiwi

the shows get a ID similar to SHxxxx00002 (xxx is different for each title) the problem is all shows with the same title all get the same id so will only get recorded once if i mark one of them as watched they all get marked as watched.

What version of SageTV are your using? Because in all my tests, as soon as you have "SH" prefix and the show doesn't have an episode title, if you mark it as watched it only mark the current airing and not all the shows with the same ID.

Also it may exists a property in the sage.properties file that change this behavior but I can't find anything nor remember anything about this. You may contact SageTV or post a new thread for this.

Regards,
Stéphane.

[ekiwi]

ok there is a feature in setup/detailed setup/Customize called

"detect repeat airings with non unique ids"

the help for this is as follows

some airing have non-unique ids, causing sagetv to be unable to easily distinguish whether they are actually all the same show or different episodes of that show

do you want sagetv to perform additional checks to attempt to recognize when such shows are all the same so that they dont all get recorded

I had this turned on when i turn this off i appears to work now.

thanks stephane and john

[owilsky]

Hi Stéphane,

I am the author of a tool that creates xmltv files for Germany (see my signature).

I love your importer and use it at home.
Now I have some questions to improve my tool:

1.
This is more a question concerning your java class files:
How do I get a line break into the description of SageTV?
I now have a CR/LF in my xmltv files although I know that that's not really conform to the xmltv standard.

2.
What happens if I do an import of three weeks of xml data today and the same in one week?
Will the middle week be imported a second time if ...

a. if the EPIDs/ MVIDs/ SHIDs have been changed since then?
b. if they did not change, but e.g. a title or parts of the description has changed?

And what happens if my data provider updates its program maybe because there will be soccer tonight which was not planned two weeks ago. Actually there should have been a movie.
Will that movie get overwritten in the sage DB?

I ask because I often see that if there is a conflict, SageTV will import nothing and the EPG reports "no data".

Thanks and keep up the great work,

Oliver