Custom made EPGDATA for Asia

[redman]

Well,
It has taken me many many MANY hours of stuffing around.
Hundreds of webpages and forums searched.

But finally, I have created a VB page sniffer for Malaysia which dumps listings into a MSSQL database, does some episode numbering and logic, then extracts out into a epgdata.xml file.

Works great. Plus in the MSSQL I have the IMDB database, so I can cross reference movies and such.

Which leads me to my question:
How can I enable more fields in the Sage java files. I want stuff such as episode-num, and actor, country, mpaa, etc that is possible from XMLTV, not just the title / sub-title / desc that is availablenow.
Anyone have anything on this?


In regards to finally getting the guide working:
I had major problems once I had created the epgdata in XMLTV format.
When trying to load the guide, it would just hang saying 'Importing XMLTV' or something like that. Well, after fiddling around with it, it seems to have problems with some characters in the guide (such as '&') or carriage returns.
Once I got rid of the '&' it imported sweet.

Now I have an EPG for KL, which will be pretty easy for me to change when i mve cities (i.e. to Singapore later this year)

Thanks everyone at this forum for the help and knowledge that is contained herein, as well as the guys at www.nerdthings.com who game me the idea and bits of code for the scraper.

Now I just have to get the damn OSD on the PVR-350 working on PAL and I can laydown and enjoy.

[nielm]

There is an alternative XMLTV plugin that gets actors/directors:
http://forums.sage.tv/forums/showthr...&threadid=2234
as for other stuff, you will have to add it yourself, but the Sage DB has no intenal fields for most of the things in XMLTV, so you will have to add it to the description. (exception: EPGID could contain epsiode num , if you can be sure it will be globally unique).

[MillHHaus]

Hi there,

Saw your post on your EPG that you created for Malaysia and want to know if you have done the same for Singapore, as you mentioned.

I am trying to get an EPG, but arent having alot of success.

Any assistance you could give me would be muchly appreciated!

Thanks.

David.

[redman]

There is a great epgdata.xml available from the guys at www.htpc-gear.com
It includes actors, categories and Epside names where available.

You can find it here:http://www.htpc-gear.com/epgdata.xml

It updates each night about 3am

Have fun.

[Saint71]

Hi Redman, any chance you'd make your scraper available to the public? I've been messing for monthstrying to scrape the program listings of astro, but no luck so far

Another KL dweller,

Peter.

[redman]

Hi Saint,
I have since moved to Singapore where the epg is better, but the signal quality worse. (and don't get me started on comskipping - damn Singtel doesn't have ONE black frame, and the logo never dissapears!!...sigh)

Therefore the state of the scraper I was using is quite neglected.
There was also the other problem, in that I had to learn how to do it from scratch, and in the end the coding was crap (to put it bluntly)
I learned a lot, which is always good, but basically made use of the tools I knew a little bit about. In hindsight, there are much much easier ways of doing it.

I have attached the files I was using. The way I had it, was a really clunky way, that in the end I ran manually one a week, beacuase sometime it would fail and drop out.
It uses a SQL database to write each of the listing in the scraper, First the show titles, times, etc then it goes back and gets each of the program details, then the last part creates the XML file.

It is quite fiddely and like I said, wasn't working that well. But at the end of the day I had at least a weeks worth of epg data in sage which made it work.

The table creation script

The vbs file I placed in the inetpub/wwroot directory to call through IIS

I called the script through this file

This will load a web page while it is running. If it fails it will display a error page, just run it again until it gets ALL the data.


Hope you can make use of this. At least the logic of the scraper.
Let me know if it gets working.

[Saint71]

Thanks so much! I will dissect it later tonight and see if I can get my head around it. I'm by far no guru coder either. It's too bad someone from our region doesn't join in on the xmltv scraper project and make a malaysia/singapore version. I'm not using sagetv currently, but snapstream (which also reads xmltv data). I've been looking at sage for awhile now, and it looks interesting. Do you have any comments on why you choose sage instead of snapstream? Or Beyond TV as it's called now, silly name.

Cheers!

[redman]

The htpc guys (Skateni) has a fantastic job done with Starhub in singapore for the epg, totally handoff... awesome.

I chose sage for 3 very simple reasons:
> PVR-350 TV out with OSD for pic quality
> Studio / ability to modify and change out stuff (nielm's STV is pretty good at this anyway)
> because i liked it better.

Have fun.

[Saint71]

I grab the vga signal from the gfx card straight to the projector, so I don't really need the dedicated tvout of my 350... That studio thing in sage sounds good, I'll have a look at sage... Snapstreams latest builds feels very laggy and I'm having a lot of problems with it changing channels on my astro box, how did you go about that? I'm using the actisys, thinking of somehow sending the channel change thru rs232, but not sure if it can be done on the box I got from astro, it does have a 232 port, but can't find any data on how to use it.

[redman]

I had no luch with the RS232 on the astro box. I bought a USB-UIRT and it work great.
Especially now that i don't really watch 'live-tv' anymore. I don't have to worry about the few seconds it takes to change the station.
The biggest headache is that astro never starts or finishes the shows on time, so you have to keep adding padding to get the end of shows.

[silkshadow]

This was created by a genius and posted over in the snapstream forums http://home.so-net.com.hk/~bdickens/TVxb/. I am just starting to look into this HTPC stuff now (comming from multiple Tivo land) but so far TVxb looks incredible. I am in the Philippines so I will be monkeying around with those ini's but the creator concentrated on Singapore so, I think, should fufill your needs.

[Saint71]

Redman: Yeah their timing is really off sometimes, I pad my recordings 3 mins, but even with that I sometimes miss the endings on different shows.

When you used your USB-UIRT did it change the channels perfectly for you, never any misstakes? I frequently get screwups when it sends a signla to the astro box, i.e, channel 13 becomes channel 1 etc... Major pain. I'm using the Actisys however... Hooks up thru serial.. maybe theUSB-UIRT does a better job..?..

Silkshadow: Thanks for the link, checking it out now!

[silkshadow]

My pleasure. If you get it working please post .