TV.Com to .My File Generator (MyScraper)

[pdavis]

I've written a small C# application for Windows that will parse the source file from tv.com and generate a .my file that you can then save. Feedback is welcome.

Simply download the My Scraper Executable, unzip it, and run the MyScraper.exe file. For instructions go to the Help->Instructions menu option.

* My Scraper Executable
* My Scraper C# .Net Source Code

Just for a little practice I converted the application to a web app...

http://pdavis.somee.com/

It does mostly the same thing as the Windows application but may be a version behind and is not nearly as easy to use... plus no instructions... but instructions can also be found in the executable folder in the Instructions.htm file.

For the latest release I have added illegal character checking and added a missing file to the Executable pack.

[millers_35]

Very nice pdavis, I will be using this extensively. Greatly appreciated.

Miller

[dvd_maniac]

I've done about a dozen complete series in almost no time and effort at all.

This program rocks. I hope it will grow now that metedata properties files are here. Unlike .my files, when you create a properties file for an import in actually adds that info into the wiz.bin and becomes searchable.


Thanx pdavis

[millers_35]

Yeah its so quick and easy!!! I had a couple series I was dreading doing but now I got already got em down.

Miller

[roxy99]

Looks great and I'm looking forward to trying it, thank you. Does the application fetch the data and parse from TV.com automatically or do you first have to manually search and download from TV.com for tv series?

[roxy99]

Works like a charm. Thanks for that. All that's remaining is to incorporate this as a Sage plugin and get the plugin to find the exact URL of the episode within TV.com.

[dvd_maniac]

Originally Posted by roxy99

Works like a charm. Thanks for that. All that's remaining is to incorporate this as a Sage plugin and get the plugin to find the exact URL of the episode within TV.com.

I disagree that all that is remaining is Sage plugin support for tv.com.
I really hope that someone can incorporate metadata .properites file support as well. To have .my files for a show with say 200+ episodes makes it easy to find out what a particular episode is about, but if you are trying to find a particular episode of a series and you know what the episode is about but do not know the episode name than you have to go through them one at a time and display "Video Details" of each until you find it. With .Properties files they merge with the wiz.bin and become searchable from Import Videos Search Screen (With Custom Search STVi). So if I wanted to search for all Star Trek episodes that deal with Klingons I could search the keyword "Klingon" and find them. With .my files we can't do that.

[Belgrath]

Originally Posted by dvd_maniac

I disagree that all that is remaining is Sage plugin support for tv.com.
I really hope that someone can incorporate metadata .properites file support as well. To have .my files for a show with say 200+ episodes makes it easy to find out what a particular episode is about, but if you are trying to find a particular episode of a series and you know what the episode is about but do not know the episode name than you have to go through them one at a time and display "Video Details" of each until you find it. With .Properties files they merge with the wiz.bin and become searchable from Import Videos Search Screen (With Custom Search STVi). So if I wanted to search for all Star Trek episodes that deal with Klingons I could search the keyword "Klingon" and find them. With .my files we can't do that.

I'd like to see the follow options if possible.
- Having the option to select which fields from the Tv.com website I'd like to place in my MY file.
- File syntax preference, so that I can name the files in the proper format. <Series> [<season>x<episode>] <Title>.my
- I would like to be able to get the data from Episode Recap which tends to be a bit more detailed, and place that into the <TV Overview> field.

Those options would really be a great added benifit. Thanks much ahead of time!!

Belgrath

[alon24]

Originally Posted by dvd_maniac

I've done about a dozen complete series in almost no time and effort at all.

This program rocks. I hope it will grow now that metedata properties files are here. Unlike .my files, when you create a properties file for an import in actually adds that info into the wiz.bin and becomes searchable.


Thanx pdavis

Can you put a link to what are the metadata .properties?

[dvd_maniac]

The syntax/Format is a little different from the .my file format.
I will attach a metedata file here:

[Opus4]

Originally Posted by alon24

Can you put a link to what are the metadata .properties?

Recording metadata properties files were added in the v6.1.4 beta, but that was just a start. It's property support will probably be filled out more in future versions (not 6.1)

- Andy

[justme]

Sorry for the late response...

But I want to thank you for this app and especially the new version, with handling of illegal characters. It's easy, pretty fast and highly usefull.

Wishlist:
1) Another vote for allowing output as a .properties file. I'd like it to be user selectable. ie create .my or .properties OR both

2) Another vote for adding optional parsing of episode recap descriptions. I personally wouldn't want it to replace the current/normal episode summaries when browsing thru my media library. Still it would be nice to add the data under another metatag. Maybe one that could be accessed via a STV mod(of course the data would have to be in the .my file first), if the user wanted the more detailed recap.

PS:I don't know if I'm providing info people already know or if it can even be put to any use but... you can simplify the URLs once you know the show number tv.com uses for the series you're interested in. Also you can get a episode list of a season by using season=X (where X=the season number) or 0 for all seasons, instead of all that dropdown junk that normally ends up at the end of the URL if you browse there by clicking links. ie

the normal link for a ep listing of StarTrek:Voyager season 3 looks like this:
http://www.tv.com/star-trek-voyager/show/274/episode_listings.html?season=3&tag=nav_bar;3
Once you know that TV.com's show number for Voyager is 274 and knowing that we don't need all the drop down junk in the URL means that you can shorten it to this:
http://www.tv.com/show/274/episode_listings.html?season=3
You could just change that 3 to a 0 to get a ep list for all seasons.

The same logic also seems to apply to an episode of a show, once you get that episode's tv.com number. For example episode 29 of ST:Voyager uses the number 10668, so the episode 29 summary link of:
http://www.tv.com/star-trek-voyager/prototype/episode/10668/summary.html?tag=ep_list;title;28
can be reduced to:
http://www.tv.com/episode/10668/summary.html
You can also access the episode 29 recap with this simple link:
http://www.tv.com/episode/10668/recap.html
It would be nice to be able to format a link to get an episode summary for a certian episode without knowing the TV.com ep number. For instance get a summary for episode 29 without knowing it's ep number is 10668. But I haven't found a formatting that does that. Maybe smarter people than me can work out if that's possible.

[roxy99]

The avi container can support text information so why can't a tag be inserted into the avi that gives the program info? Then a sage plugin can read the tag information directly from the avi instead of having to clutter the hard drive with all these .my files.

Dvdmaniac,

How is replacing one text file format for another text file format any different? I mean syntax of .my text files vs your metadata text file is different but so what? What is the advantage?

[dvd_maniac]

The advantage is the Syntax and the way that Sage currently handles the two files.

The .my files are currently not searchable inside of Sage.
I have every episode of Star Trek and it is great to have the .my files for them but if I wanted to watch a particular episode with Klingons in it then how would I find it?
I would have to go into each Imported Video and bring up the Video Details and read it to find out if it was the episode I wanted.

Now the metadata .properties files import the information directly into the Wiz.bin and with the custom search stvi the info becomes searchable.
So I would just search for klingon and every video that has Klingon in the Title or Summary field will show up.
Also great for searching if you have a large Imported video folders. I have over 1500 movies and it is great to search for a particular actor and see all the movies I have with that actor in it.

The other reason is standardization in syntax. .my files can be formatted in various ways and so display the information differently. metadata .properties files(I think) are imported and syntaxed in Sage's native format and as such can only be displayed one way.

[roxy99]

Originally Posted by dvd_maniac

The advantage is the Syntax and the way that Sage currently handles the two files.

The .my files are currently not searchable inside of Sage.
I have every episode of Star Trek and it is great to have the .my files for them but if I wanted to watch a particular episode with Klingons in it then how would I find it?
I would have to go into each Imported Video and bring up the Video Details and read it to find out if it was the episode I wanted.

Now the metadata .properties files import the information directly into the Wiz.bin and with the custom search stvi the info becomes searchable.
So I would just search for klingon and every video that has Klingon in the Title or Summary field will show up.
Also great for searching if you have a large Imported video folders. I have over 1500 movies and it is great to search for a particular actor and see all the movies I have with that actor in it.

The other reason is standardization in syntax. .my files can be formatted in various ways and so display the information differently. metadata .properties files(I think) are imported and syntaxed in Sage's native format and as such can only be displayed one way.

That makes sense to me now. I did'nt realize that Wiz.bin incorporates the metadata .properties file. Pdavis should'nt have much problem implementing that as an additional format.

[roxy99]

I have to say you're not getting the props you deserve. I'm learning Python programming and as an exercise I am trying to reverse engineer the work you did. Needless to say, I am having a bitch of a time parsing TV.com's html.

I have to give you kudos for a job well done. I'm sure you faced many challenges in parsing the data. If you can share any tips with me, feel free to pm me.

Thanks again for this great tool.

[Armondole]

Originally Posted by Belgrath

I'd like to see the follow options if possible.

- File syntax preference, so that I can name the files in the proper format. <Series> [<season>x<episode>] <Title>.my

Belgrath

I agree with you on this Belgrath. This would be a very handy feature if it was possible to implement.

Thanks for the great tool!

[Armondole]

Has anyone ever used the XboxMediaCenter TV.com scraper for TV/Movies? I find that it is extremely accurate and easy to use... I really know nothing about how this stuff works, but I thought this might be helpful info for anyone trying to improve this script.

http://www.xboxmediacenter.com/wiki/...le=Scraper.xml

[pdavis]

Thank you so much for all the great feedback. I will take your suggestions and incorporate those I can. I'm also planning on making an export to XML option now that we can import XML information back into Sage through the web interface. I also noticed that the episode guide page has changed on TV.com so that functionality no longer works, I will update that as well.