New XMLTV Import Tool

[StephaneM]

Originally Posted by svalmorri

I think there is an issue with the latest version of the importer, my kid let me notice it. It looks like several different shows are treated as the same show appearing in the EPG grid with the same description.

I tested your XMLTV file and all is fine. So make sure you have selected the right diffrentiation method in the Importer.

Go to the XMLTV File and Grabber menu, edit your XMLTV Source and in the third screen choose "Based on episode title or program description".

Also, if you have not wiped out your old computer, I would suggest that you uninstall the Importer on your new PC, install the same version as you were using on the old one. Get back you settings for the Importer from the old PC and copy them to the new one (this is the Config & Logs folder you have to copy, C:\Documents And Settings\All Users\Application Data\LM Gestion\SageTV XMLTV Importer) and then upgrade to version 3.x

The file you want to absolutely keep from your old install is the lookup.xml file as it contains all the Importer history (EPGID / Original air date). Without this file, your old wiz bin will not be that helpful.

Regards,
Stéphane.

[svalmorri]

Hi,

my settings were correct, same as per your recommendation. I loaded my former lookup.xml and re-run the import but had the same result: all the Simpson episodes were marked with the same description.

At this point something captured my attention, and here is how I fixed it. I noticed that these shows were marked as Movie and set as first runs for all the occurrences. I then edited Categories and Ratings and noticed that I had selected a keyword that made these shows fall under the movie category. Removed this extra keyword and re-run the importer: voilà, they are not movies any more and each episode has now its correct description.

Perhaps should I have fixed this also by setting the checkbox "shows of this category are not the same on all channels" for the Movie category?

Thanks for your time!

[StephaneM]

Originally Posted by svalmorri

At this point something captured my attention, and here is how I fixed it. I noticed that these shows were marked as Movie and set as first runs for all the occurrences.

Yes, I should have noticed it. The description is not used for Movies or show with episode tilte/numbers. That's why the Importer was seeing all episodes as the exact same movie.

[planetc]

Stephane, can you tell me, is it possible to upgrade to the donator version without losing my settings? I set up the freeware version to test which has been running for a while now and is very good, but I would like to take advantage of the scheduling functions of the donator edition.

[StephaneM]

Originally Posted by planetc

Stephane, can you tell me, is it possible to upgrade to the donator version without losing my settings?

Yes. However, it is recommended that you uninstall your current version (responding "No" to the "Do you want to remove all your settings?") and then install the newer version.

Regards,
Stéphane.

[planetc]

thankyou Stephane

[Whitey]

I've had a strange problem occur over the last few days. Alot of my channels that usually have no data are all showing the same information from one channel. It's even affected some channels that only broadcast part of the day. During broadcast hours the info is fine. Its like the "no data" info has been corrupted somehow

Whitey

[StephaneM]

Originally Posted by Whitey

Alot of my channels that usually have no data are all showing the same information from one channel. It's even affected some channels that only broadcast part of the day. During broadcast hours the info is fine. Its like the "no data" info has been corrupted somehow

You'll have to send me your XMLTV file (and possibly the content of your config & log folder) zipped at lmgestion@hotmail.com and let me know what channels are messed like this.

Regards,
Stéphane.

[rickgillyon]

Hi Stephane, I don't know if you spotted my posting elsewhere, but in the new version I have shows appearing as first runs that have the previously-shown tag present in the xml. Any idea why this would happen, would the importer override previously-shown because there's nothing in the history? Thanks!

[StephaneM]

Originally Posted by rickgillyon

I have shows appearing as first runs that have the previously-shown tag present in the xml. Any idea why this would happen, would the importer override previously-shown because there's nothing in the history?

The Importer can override previously-shown especially when the tag doesn't carry a date. The Importer will lookup its history to find the original air date.

If you want you can send me your XMLTV file and the content of your config & log folder zipped to lmgestion@hotmail.com and I'll take a look at it.

Regards,
Stéphane.

[rickgillyon]

Okay, I'll email locations of the files (a bit big to email themselves), but it may not be worth your time if it's the expected behaviour. I was expecting that if the XML flagged as a repeat it would be a repeat whatever - I understood that if it matched a historical airing, a "first run" would be overridden, but didn't expect the reverse. I assumed that the importer would believe the XML that it had been on before, even if it were outside the history...

Thanks for the help!

[StephaneM]

Originally Posted by rickgillyon

I was expecting that if the XML flagged as a repeat it would be a repeat whatever

Unfortunately your XMLTV file has the same show "New Tricks / Old Dogs" on UKTV Drama without any information about being a first run / re run. So by default the show is a first run for the Importer. The other instance of the show despite being a re run, doesn't change that as the Importer lookup the orginal air date of the show (as the previously-shown node doesn't have a date).

The only thing that could be improved is that the first "new tricks/old dogs" occurence for the Importer is the 11/10/2008, but in this case the show is also aired on another channel/lineup one month before. I don't know however if it is worth it for the Importer to check that if the original air date is always older than the currently imported show air date (usually you import 7 to 14 days at a time in the Importer, so for first run / re run, 7 days doesn't count)

If you can import only 7 to 14 days it would be much better (after 14 days, guide data is often unreliable and incomplete, so you give the Importer some wrong data that will "pollute" its history)

Regards,
Stéphane.

[rickgillyon]

Thanks for the explanation, it's obviously a complex set of algorithms to sort this out. I'll try to limit my import, seems like a good plan (my XML file is now about 85MB as you saw).

Not sure if it makes a lot of sense for me though to have the program on 15th Sep showing an Original Air Date a month later. Seems like if it processed them in date order it could mark the first as a repeat, with a dummy Original Air Date if required, then would find the others also as repeats as matched with the first. I have no idea how your program works though, I'm sure there's more to it than that!

[crollorc]

hi stephane,

my grabber batch file doesn't appear to complete. it's been doing this for quite some time and so i have to run i by hand.

when i run it interactively it takes a few minutes but in the importer trace log i see -

20080911 00:00:06.245 - Info - Import Started
20080911 00:00:06.964 - Info - Launching XMLTV grabber 'D:\sagetv\xmltv-0.5.51b-win32\local.bat'
20080911 00:00:13.432 - Info - Processing XMLTV Source: XMLTV

which seems to indicate that it exits after 7 seconds.

the batch file itself looks like -

d:
cd \sagetv\xmltv-0.5.51b-win32
xmltv.exe tv_grab_uk_rt --output D:\sagetv\etc\xmltv-data.xml

i'm running the latest version of your import tool.

thanks for any help!

colm

[StephaneM]

Originally Posted by crollorc

my grabber batch file doesn't appear to complete. it's been doing this for quite some time and so i have to run i by hand.

I would not use the batch file, but instead directly type grabber and arguments in the Importer.

It may be that the batch file is not seen as a "regular" process that the Importer can wait after before resuming.

So grabber would be: d:\sagetv\xmltv-0.5.51b-win32\xmltv.exe
args would be: tv_grab_uk_rt --output D:\sagetv\etc\xmltv-data.xml

Regards,
Stéphane.

[planetc]

I am unsure and wish to clarify please. I have set the importer to run my dgxmltv grabber automatically which it doesn't appear to do. I can run it manually without issue and manually running the importer is fine and will process the data that I have already grabbed. I have checked and double checked the path to the grabber and can start it using a command prompt using this path without issue.
What I am unsure of is if the importer stores the information direct from the grabber rather than updating the epgdata.xml file and processing it or does it only run the grabber when triggered by the task scheduler? If I manually start an import then it does not appear to run the grabber or update my epgdata.xml source file, nor can I see any reference to the grabber in the logs?

[rickgillyon]

Got to say, great job with this software. Finally all my within-a-week repeats aren't all recording, looks very steady so I made the donation (despite not really needing the scheduling). Very impressive, a big improvement on my previous importer.

Just the one small problem, with the non-date-order processing, but hopefully that problem should crop up only rarely.

Nice one!

[StephaneM]

Originally Posted by planetc

What I am unsure of is if the importer stores the information direct from the grabber rather than updating the epgdata.xml file and processing it or does it only run the grabber when triggered by the task scheduler?

The Importer only run the grabber during scheduled updates (so not when you click on the "Start Import" button). The Importer doesn't get the information directly from the grabber, instead the grabber update the XMLTV file and the Importer load this file when the grabber process terminates.

nor can I see any reference to the grabber in the logs?

When the grabber is launched during automatic import it is written in the logs.

Regards,
Stéphane.

[StephaneM]

Originally Posted by rickgillyon

Just the one small problem, with the non-date-order processing, but hopefully that problem should crop up only rarely.

This has been taken care of for the upcoming version.

[Jabroni]

StephaneM was wondering if you have never considering implementing the way the DVBDream Network Plugin of mfury uses to insert the EPG data onto SageTV. He developed a module for sagetv that listens on a socket, and receives the updates via file parameter or directly thru the socket. This would be good for people like me that still want to use the builtin sagetv epg for some tuners/channels. I was thinking into building my own xmltv parser.. but I thought it would be more practical if we could manage to get this built onto your same parser...

Heres what mfury posted me on the DVBN thread of his plugin

To import EPG entries into SageTV via the DvbEpgImport module, messages are sent through a standard,
"stream" based socket. Commands are sent as a single line of text, terminated by a CR/LF character
combination (\r\n). After each command is processed, the module responds with "OK\r\n" on the
socket connection. If an error occurs, the module responds with "ERROR\r\n". Here are the commands
that can be sent:

1) To start processing a new set of EPG entries:

START <lineup>|<filename>\r\n

Where:
  <lineup> = The SageTV lineup where the entries will be imported.
  <filename> = (Optional) The name of a file containing epg entries to import.

2) If a filename was specified on the "START" command, entries will be read from that file. Otherwise
   individual epg entries are sent as tab-delimited strings, terminated by "\r\n", containing the
   following fields:

Channel Number - The "physical" channel number in Sage.
Channel Name - The channel name (only used if the channel is automatically created).
Sid - The sid number from the satellite (not used).
Event Id - The "event id" from the satellite (not used).
Show Id - The "show id" to use in Sage. (This is the unique id that Sage will use internally to
  identify the program.)
Start Time - The event start time in HH:MM:SS format (24 hr clock).
Duration - The event duration in HH:MM:SS format.
Category - The category and (optionally) sub-category of the event. (Specified
  as "category/sub-category".)
Rating - The rating for the event. (i.e. "PG-13", "R", etc.)
Year - The year the event first aired.
L - Detailed rating flag for "Language" ("1" or "0").
GL - Detailed rating flag for "Graphic Language" ("1" or "0").
N - Detailed rating flag for "Nudity" ("1" or "0").
SSC - Detailed rating flag for "Strong Sexual Content" ("1" or "0").
V - Detailed rating flag for "Violence" ("1" or "0").
GV - Detailed rating flag for "Graphic Violence" ("1" or "0").
AC - Detailed rating flag for "Adult Content" ("1" or "0").
HD - Flag for HD content ("1" or "0").
CC - Flag for closed captioning present ("1" or "0").
Stereo - Flag for stereo sound ("1" or "0").
Event Name - The event name (or series title).
Short Description - The "short" description for the event (or episode title).
Long Description - The "long" or detailed description for the event.

3) When all entries have been sent, the following command is sent to close communications:

END\r\n

Notes:

- When reading entries from a file, the session ends when the file has been processed, and
  it is not necessary to send the "END" message.
- Long entries can be broken up into more than one socket message, by appending the "~"
  character to all but the final message. In testing, I found from DVBDream, it was necessary
  to break up entries longer that 210 characters into multiple socket messages, otherwise
  DVBDream would crash. (Some sort of memory problem, I think.)

What do you think of these posibilities??