New XMLTV Import Tool

[StephaneM]

Originally Posted by rsagetv99

Don't forget to remove the "." that is before the last 4 digits of the ID, it appears that this is what the internal EPG does so unless you do it to then the IDs won't match.

Yes, this how I have done it, and it works well for now (I even have it process the HDTV flag and pass it to SageTV natively), you'll be able to download this new version (1.3) on Friday.

Note: there is also some EPGID with another dot after the last four digits for two parts show, this one is removed also (as well as the last part so that the EPGID is always 12 chars long).

Regards,
Stéphane.

[nielm]

Originally Posted by svalmorri

Is there any way in the importer or in SageTV to virtualize the channels to appear with different and more confortable numbering? Example: RAI1 is channel (RF) no. 63, I would like it to show as no. 1 in the lineup and in the EPG.

Is it possible? I have found some information in the former posts but it's a bit confusing.

not in Sage, or with any importer, but you can in Windows: You can override the channel number->frequency assignments so that you can have a custom channel lineup...

http://forums.sage.tv/forums/showthread.php?t=2042
http://www.prize.nl/software/freqcopy
http://forums.sage.tv/forums/showthread.php?t=12068

[nielm]

Hi stephane,

How does the importer cope with generating EPGID's for the following scenarios:

A movie split into 2 parts (XMLTV data will show the movies Title, no sub-title and will show it in 2 time-slots) (Hopefully it will not use the same EPGID for both timeslots)

A show where each 'epsiode' is split into 2 halfs and shown on subsequent days: the XMLTV data have title and sub-title, but the sub-title for the 2 halfs is the same...

A show where the sub-titles of the episodes are only shown as: "1/12", "2/12" etc... When a new season of this show is shown, the same sub-titles are re-used (!).

These 3 scenarios have caused me problems with the choice of EPGID -- leading to episodes not being recorded, or both the original episode and the repeat being recorded (depending on which importer I use)...

Thanks,
Niel

[StephaneM]

Originally Posted by nielm

A movie split into 2 parts (XMLTV data will show the movies Title, no sub-title and will show it in 2 time-slots) (Hopefully it will not use the same EPGID for both timeslots)

If your XMLTV file contains either a part number (xmltv_ns system) or a different description, the Importer will generate two different EPGID for each part.

A show where each 'epsiode' is split into 2 halfs and shown on subsequent days: the XMLTV data have title and sub-title, but the sub-title for the 2 halfs is the same...

Your XMLTV file must have the full episode number or at least the correct part number (xmltv_ns system only)

A show where the sub-titles of the episodes are only shown as: "1/12", "2/12" etc... When a new season of this show is shown, the same sub-titles are re-used (!).

The same EPGID will be re-used for the new season. The Importer has no way to know it is a new season.

My Importer will always try to re-use an EPGID if it can, based on what it can find on the XMLTV file :

* The program title
* The Episode title
* The Episode number (xmltv_ns system only / onscreen is only used to "decorate" the episode title if found)
* The program description OR air date (user choice)

When the Importer found an episode title or an episode number, it will not use the program description to try to identify the program. However the Importer will always try to "match" the episode title and episode number and in case of inconsistency will correct the situation.

So if your XMLTV data are "good" the Importer will do a wonderful job, otherwise it will only try to be smart .

Note: you can always contact a grabber author for improving the grabber. It is always better to try to have clean XMLTV data (compliant with the DTD) than having the XMLTV Importer to do insane things to replace / create mising informations. Moreover the grabber author does have a great advantage : it knows the data better than I would ever do.

Regards,
Stéphane.

[Alfiegerner]

Hi Stephane,

I would love to get this going and have tried many times now. Currently I can get through to the stage where I choose my lineup. Sage then says please wait while it imports data and then it simply hangs at that screen. The xmltv.log suggests everything is ok:

04-05-2006 19:24:44,496 Reading configuration
04-05-2006 19:24:47,339 Start EPG update
04-05-2006 19:24:47,339 Reading configuration
04-05-2006 19:24:47,339 Reading Channels File
04-05-2006 19:24:47,370 Creating channels array (0)
04-05-2006 19:24:47,370 Setting SageTV lineup
04-05-2006 19:24:47,370 Reading Lineups File
04-05-2006 19:24:47,370 Reading Programs File : Programs1.xml
04-05-2006 19:24:47,542 Adding programs to SageTV DB
04-05-2006 19:24:52,230 2343 programs added
04-05-2006 19:24:52,230 Reading Episodes File : Episodes1.xml
04-05-2006 19:24:52,339 Adding airing to SageTV DB
04-05-2006 19:25:00,667 4217 airings added
04-05-2006 19:25:00,667 End Update

However, Sage is still stuck on the "importing data screen"? Might this be caused if i did not have a dtd file? The xmltv grabber I am using does not supply one.

edit: Just noticed if I go back and then back through the channel line up screens I get to the Basic or Extended Basic Service options. If I choose one and I then get through the channel selection screen although there are no channels.

[rsagetv99]

Originally Posted by StephaneM

(I even have it process the HDTV flag and pass it to SageTV natively).

This is great! Maybe in a future release you could implement a way for HDTV channels to be identified and separate favorites setup for HD shows on HD channels that you receive.

Currently I have a script that will read a channel listing of my HD channels and prepends the HD show titles on those channels with "HD -". This way I can setup a Favorite for the HD version of a show or if my HD setup is shaky I can setup a Favorite for the HD and SD versions. It also gives the advantage of being able to search for shows beginning with "HD -" so I can see all of the HD programming coming up.

Demedius implemented something similar in his importer but I have not been able to get it to work.

[StephaneM]

Hello,

Originally Posted by Alfiegerner

The xmltv.log suggests everything is ok

Well, the xmltv.log appears to be fine but no channels are loaded. So it seems that you have not properly edited your channels in the Importer Channel Editor.

You should check that your lineups and channels are ok in the Channel Editor:

* First create your lineups. I suggest that you create one lineup for each TV source you may have, for instance one for Satellite, one for Antenna etc...

* Then review the channels list, you'll see here all channels found in your XMLTV file. For each channel, assign a name, call-sign (a short name), channel number (used to tune the channel), and choose the lineup to put the channel on. If you forget one channel, your lineup will not get saved properly.

The easiest way is to begin with one lineup, test the result in SageTV and then adding the another lineup in the channel editor.

Might this be caused if i did not have a dtd file? The xmltv grabber I am using does not supply one.

No the Importer doesn't care, it just want a valid XML file that is compliant to the XMLTV DTD but it doesn't have to make reference to the DTD. If your XMLTV file was making reference to the DTD and the DTD was not found, then the Importer would complain.

Regards,
Stéphane.

[StephaneM]

Originally Posted by rsagetv99

This is great! Maybe in a future release you could implement a way for HDTV channels to be identified and separate favorites setup for HD shows on HD channels that you receive.

I am not yet familiar with SageTV. However it would seems that as the HDTV flag is in the guide, that it would be better that SageTV adress this kind of feature.

<offtopic>
I guess this "favorite" thing is confusing. I would rather see SageTV have a "Season" pass, and a "Favorite" recording. The "Season" pass should be based on EPGID (and not on the title as it is today) and the "Favorite" should be based on show properties, like "is a sci-fi movie on channel 16", or is "CSI in HD" etc...

If we were on BeyondTV, I would have done this in a Web Plugin. But I don't think I am that good in Java to make it work with SageTV.
</offtopic>

Regards,
Stéphane.

[Alfiegerner]

Thanks for the help Stephane. I will try again with just one lineup. Some queries about the call-sign paramater.

1. Do they need to be unique for each channel?

2. Also, if you use two lineups, do you need to give each channel the same call-sign in the additional channels section (for the 2nd lineup) as it has in the first lineup?

[StephaneM]

Originally Posted by Alfiegerner

1. Do they need to be unique for each channel?

It's not mandatory, but it makes more sense. The call-sign is mainly significant for USA/Canada (and maybe other countries), so for Europeans it is really just a short name / another name for the channel, if you want you can make it the same as the channel name.

For instance for "Cartoon Network", I use "Cartoon" as the call-sign, you can use "CN" or whatever you want.

You'll see these call-sign everywhere in SageTV, and mainly in the guide, so if you have the same call-sign for all your channels it will be a little difficult to know which channel is which.

2. Also, if you use two lineups, do you need to give each channel the same call-sign in the additional channels section (for the 2nd lineup) as it has in the first lineup?

Again not mandatory but it makes more sense. The additional channels tab usually contains :

* duplicated channels, for instance you can receive "Cartoon Network" on your "Cable" lineup and also on your "Satellite" lineup, so you'll have to duplicate "Cartoon Network" in the additional channel tab. When duplicating it is capital that you use the same ID as the original channel, and preferable to use the same name and call-sign, then type the required channel number for the lineup, choose the lineup to put the channel on and select the original channel from which the programs will be duplicated.

* dummy channels : the channels you receive but that are not in the XMLTV file. You'll have to made up a channel ID and fill in the other fields as you want.

* replicated channels, it is a duplicate channel with a different ID as the original one. You will use this when you have one channel that also broadcast exclusively in HDTV. For instance : "Cartoon Network" and "Cartoon Network HD", both channels are broadcasting the same shows on two differents frequency and you want to differenciate them in your program guide. This will also in the future enable you to create "+1" channels (a channel that broadcast the same shows as the original channel but one hour later)

Regards,
Stéphane.

[Alfiegerner]

Ok got one lineup working now

It looks like the bit that is causing me problems is creating the second lineup using the additional channels tab.

To fill you in, I am importing one xmltv file. From this I want to create two lineups, one with all of the channels in the xmltv file (for cable) and one with only a few (for OTA DVB-T). My problems come when I try to make the second lineup.

I have attached some screenshots to show you how I have setup the second lineup using the two tabs. Would you mind having a look for me to see where I am going wrong?

[StephaneM]

Originally Posted by Alfiegerner

I have attached some screenshots to show you .../... would you mind having a look for me?

Of course, but where are the screenshots?
Edit : Ok now they are available.

[StephaneM]

Originally Posted by Alfiegerner

Would you mind having a look for me to see where I am going wrong?

This seems fine. The ID are matching, you choosed the right lineup. Now I can't tell you if the channel number you choosed for each channels on lineup 2 is supposed to be the same as in lineup 1.

Usually with DVB-T the channel number is either:

* the number found in the .frq file create by SageTV
* the channel number in SBDARecorder
* the favorite number in MyTheatre.

So now you just have to click on the "Update" button, this will close and save your channels list. To be sure all was saved properly, open the channel editor again, you'll immediately see if all was saved (the channel editor can be mean to you : if it shock on an error it is supposed to show it to you in the grid and not close itself)

When you know that all was saved properly, click on the "Import" button on the main window (or if using the Wizard just click on the "Next >" button until finished).

Then go in SageTV and you'll see your two lineups.

Regards,
Stéphane.

[Alfiegerner]

Thanks for all your help Stephane but unfortunately I still can't get it to work with the second lineup. I had to switch back to my old xmltv for now so my wife gets to watch some TV but will have another look over the weekend to see what the problem is.

[rsagetv99]

Originally Posted by StephaneM

I am not yet familiar with SageTV. However it would seems that as the HDTV flag is in the guide, that it would be better that SageTV adress this kind of feature.

You would think so, asked for it a couple of times but it fell on deaf ears. No biggie, I can still use my script. I was really asking for other people since some people have problems getting kixtart setup to run my scripts.

Originally Posted by StephaneM

<offtopic>
I guess this "favorite" thing is confusing. I would rather see SageTV have a "Season" pass, and a "Favorite" recording. The "Season" pass should be based on EPGID (and not on the title as it is today) and the "Favorite" should be based on show properties, like "is a sci-fi movie on channel 16", or is "CSI in HD" etc...

If we were on BeyondTV, I would have done this in a Web Plugin. But I don't think I am that good in Java to make it work with SageTV.
</offtopic>

I think its just semantics, most Favorites are based on the title, but Sage decides whether that particular episode of that Favorite needs to be recorded based on the EPGID. This sounds like a Season pass to me.

You can also create Favorites based on categories as you mentioned, or keywords that can be found in any field. This was a big one for me since I used this feature with Tivo to automatically record my team's football games. For example, I created a keyword favorite with the words "Tampa Bay Buc* at" and had it search all fields. This would record the Buccaneer's games no matter what time or on what channel it aired. This was the main reason I chose SageTV over Beyond or any other PVR software.

[StephaneM]

Originally Posted by Alfiegerner

Thanks for all your help Stephane but unfortunately I still can't get it to work with the second lineup.../...will have another look over the weekend to see what the problem is.

Ok, if you want you can send me all your configuration files (all .xml files) in the install folder of the Importer, as well as the xml files found in the 3rdPartyGuideData folder (in the SageTV\SageTV folder) and the xmltv.log.

Also review your Event Viewer log (SageTV Utilities log) to see if there are any errors or warnings in there.

Send all these files zipped to lmgestion@hotmail.com

Regards,
Stéphane.

[StephaneM]

Originally Posted by rsagetv99

I was really asking for other people since some people have problems getting kixtart setup to run my scripts.

Yes, however I always try to limit the number of options in the Importer so as no to confuse everyone about how to set it up. For the HD flag, it can be useful for some users and will not for other so (because of Favorite). That said I don't really see why SageTV would not make use of this Flag (I don't see the point to only use for cosmetic).

I think its just semantics, most Favorites are based on the title, but Sage decides whether that particular episode of that Favorite needs to be recorded based on the EPGID. This sounds like a Season pass to me.

It is and it is not: for instance, I had set a favorite for "That '70s show" in SageTV (during my first week with Sage). Unfortunately this series is sometime mispelled in EPG "That 70's show". My Importer was not tricked by the change and correctly assigned the right EPGID. But SageTV has not recorded the misspelled show.

While it is the correct behavior for a "record all unique show with that title", it is not for a season pass where all episode are given the right EPGID whatever spelling error they may have in the title.

The same goes for "Battlestar Galactica" and "Star Trek: Enterprise" etc... (though here my Importer is tricked anyway, but Zap2it EPGID may be correct)

You can also create Favorites based on categories as you mentioned, or keywords that can be found in any field. This was a big one for me since I used this feature with Tivo to automatically record my team's football games. For example, I created a keyword favorite with the words "Tampa Bay Buc* at" and had it search all fields. This would record the Buccaneer's games no matter what time or on what channel it aired. This was the main reason I chose SageTV over Beyond or any other PVR software.

I don't use this feature a lot but I think now BTV can do this (at least you can with my Web Guide : it comes with a filter editor that enable you to create complex search criteria, like :
title begin with "Tampa" and channel is "CNN" and ((air time > 1900 and air time < 2000) Or (air date is saturday))

At first with sage I was a little confused by Favorite, I was expecting a "Record this show", "Record First run and rerun", "Record First run" and a "Keyword" recording (the Favorite). Especially because "Favorite" and "Manual Recording" (so Record this show) are not given the same priority. But as I said, I'm not yet a power user (How do you record a movie : do you set a Favorite or do you manual record it ?)

Note: you can download the 1.3 version of the Importer with the Zap2it support.

Regards,
Stéphane.

[rsagetv99]

Originally Posted by StephaneM

It is and it is not: for instance, I had set a favorite for "That '70s show" in SageTV (during my first week with Sage). Unfortunately this series is sometime mispelled in EPG "That 70's show". My Importer was not tricked by the change and correctly assigned the right EPGID. But SageTV has not recorded the misspelled show.

I see your point, you shouldn't have to but if you want to work around this problem you could just create a keyword with "That Show" and one of the actor's names. I had a related problem when I was trying to record "The Office" and it was recording the british version of the show, as you said if Sage was looking at the EPGID I probably would not have had that problem. I fixed it by creating a keyword favorite called "The Office Carell".

I don't use this feature a lot but I think now BTV can do this (at least you can with my Web Guide : it comes with a filter editor that enable you to create complex search criteria, like :
title begin with "Tampa" and channel is "CNN" and ((air time > 1900 and air time < 2000) Or (air date is saturday))

The search by keyword feature was available before, but not a Favorite or Season Pass by keyword.
Sounds like you had everything you wanted with BTV, I'm curious why you switched.

At first with sage I was a little confused by Favorite, I was expecting a "Record this show", "Record First run and rerun", "Record First run" and a "Keyword" recording (the Favorite). Especially because "Favorite" and "Manual Recording" (so Record this show) are not given the same priority. But as I said, I'm not yet a power user (How do you record a movie : do you set a Favorite or do you manual record it ?)

Most times I manually record it from the EPG, but if I am looking for a movie that is not currently in the EPG I setup a Keyword Favorite.

[StephaneM]

Originally Posted by rsagetv99

Sounds like you had everything you wanted with BTV, I'm curious why you switched.

Cause: BTV doesn't support DVB and doesn't even say something about it. So I decided it was enough for me to use more than one software for TV, and I switched to SageTV.

[kdorff]

I am running the latest Sage 5.0.2, trial.

I am evaluating BeyondTV and SageTV. I used your BeyondTV importer (and donated a couple days ago) thinking I would go with BeyondTV but now I am giving SageTV a shot. For the same reasons as I needed your importer with BeyondTV, I need it with SageTV (a custom lineup).

SageTV is running, I initially set things up with no guide but my two tuners. I found another XMLTV guide and tried it, it didn't do anything for me so I got yours.


I stopped Sage + Sage Service. Installed your importer, ran through and set it up following the docs. I started Sage. I removed my two tuners. I then tried to add a tuner. I pick my card. I select TV Tuner. I see a picture, I select TV Tuner. I say I am using it with Cable TV. I say to use "US, Canada or XMLTV". Per your instructions I selected Local Broadcast (although I have also tried Cable TV + zip, makes no difference). The software just sits at Setup Wizard: Please Select Your Local Broadcast Market, but the screen stays completely blank. The only thing I can do is press back.

I looked in Sage.properties and and it has

epg/epg_import_plugin=xmltv.XMLTVImportPluginzap2it.Zap2ItImportPlugin

I looked in the xmltv directory in SageTV\SageTV and it has the appropriate 4 class files. There is an xmltv.properties file containing

ThirdPartyGuideDataPath=C:\\Program
Files\\SageTV\\SageTV\\3rdPartyGuideData

(in one line). There is no xmltv.log file I can find, but in the directry SageTV\SageTV\3rdPartyGuideData there are 4 files that were created, all with content, when I ran the SageTV Importer.

I have tried removing SageTV XML Importer and reinstalling and all that 3 times with no joy.

Any thoughts on what I am missing?

Kevin