Matching TV shows with only Series/Episode name

[evilpenguin]

I know that BMT supports looking up shows with just the Title/Episode name since it does it for Sage Recordings, but is it possible to get BMT to do that with an imported video?

For example, i'm messing around with some new PlayOn features where I create dummy videos to represent online content and then let Sage import/scrape/organize them. For most shows off Hulu/Netflix i'm able to determine the proper season/episode information and generate an easily scrapable file name, however, some DVD sets on Netflix are done in collections of random episodes (i.e. Mythbusters) and the best I can do for those is just name them

Series - Episode.mkv

.

[stuckless]

Originally Posted by evilpenguin

I know that BMT supports looking up shows with just the Title/Episode name since it does it for Sage Recordings, but is it possible to get BMT to do that with an imported video?

For example, i'm messing around with some new PlayOn features where I create dummy videos to represent online content and then let Sage import/scrape/organize them. For most shows off Hulu/Netflix i'm able to determine the proper season/episode information and generate an easily scrapable file name, however, some DVD sets on Netflix are done in collections of random episodes (i.e. Mythbusters) and the best I can do for those is just name them

Series - Episode.mkv

.

I would have responded last night... but it's taking 2minutes to load each page on the sage forums for me... and I'm not known for patience

BMT does support finding shows by episode title (fuzz logic searching), but the challenge is knowing when to consider a file a TV show vs a Movie. For shows with Season and Episode info, it's pretty easy, because the regex will will either match or not.

The way bmt discovers tv/movie titles from the filenames is that it first runs through the filename scrapers in the STVs/Phoenix/scrapers/xbmc/ area (there is a tv and a movie sub section). It first processes the TV scrapers, and if a scraper passes, then it will use it. Once TV is done, it then processes the Movies.

For sage airings, it's fairly easy to create a tv scraper because the filename pattern for a sage recording is pretty unique (Series-EpisodeTitle-######-#), so if this matches a filename, then I assume it's in TV mode, and the scraper parses out the Series and Episode.

You can make a scraper work for your stuff, if you have it all in a TV folder. From there, you can copy/create a new scraper file and put it in scrapers area for tv filenames, and you should be all set. ie, part of your regex for the scraper will be to test that your have TV folder, etc. (make sense?)

[kmp14]

I have had the same question as I have been importing shows from tivo. I will have to dig into the xbmc folder and see if I can some stuff working. Thanks for the tips. I will report back if I have any luck.

[stuckless]

If you use the SageRecordingAiring.xml as a base, you can probably see there are 2 "functions" that you need to implement; GetShowName and GetEpisode, and remove GetAiringID.

If your TV shows are in a TV sub directory and have a filename like, "Series - Episode.mkv", then something like this may work..

<?xml version="1.0" encoding="UTF-8"?>
<!-- 
   TV Filename Scraper.  Used to Scrape Show Name, AiringTitle, and AiringId from a Sage Recording
   $$1 is always the complete file uri
-->
<scraper name="title-sageRecording" content="filename" thumb="thumb.png">
        <!--    input:          $1=complete file uri -->
        <!--    returns:        series name -->
        <GetShowName dest="3">
                <RegExp input="$$1" output="\1" dest="3">
                        <expression>.*[/\\]TV/[/\\].*[/\\]([^-]+)-([^\.]+)\.</expression>
                </RegExp>
        </GetShowName>

        <!--    input:          $1=complete file uri -->
        <!--    returns:        Compressed Episode Title from the Sage Recording -->
        <GetEpisodeTitle dest="3">
                <RegExp input="$$1" output="\2" dest="3">
                        <expression>.*[/\\]TV/[/\\].*[/\\]([^-]+)-([^\.]+)\.</expression>
                </RegExp>
        </GetEpisodeTitle>
</scraper>

BTW, when you add/modify scrapers, then you need to restart the sage server for the changes to take effect (I'm working on a better solution for this)

[stuckless]

FYI on xbmc filename scrapers...

For your expression,

<expression>.*[/\\]TV/[/\\].*[/\\]([^-]+)-([^\.]+)\.</expression>

You basically create regex groups, and in the Regex element you pull the group # that you want from the matched expression.

ie,

<RegExp input="$$1" output="\1" dest="3">

Tell the parser to get it's input from buffer $$1 (always the complete filename), and then to take the output from matched group "\1" (ie group 1), and then copy that value to the destination buffer #3.

This is not bmt's notation, but rather, in bmt, I created an implementation for the xbmc scraper engine, and then I decided to use their engine for scraping the filenames, since it provided a fairly abstract way for writing scraper functions.

[evilpenguin]

Sounds like I should just use a different separator (-- vs -) to indicate that its definitely TV and then it'll be nice and simple to make a scraper.xml file

[kmp14]

Regex always makes my head hurt...

So for each show that I want BMT to try to do its' magic on, do I need to create an xml file for each, or do I add sections to the "master" SageRecordingAiring.xml?

Also, if possible, help me out with the regex. Say I have a file named "Mad Men--We Like To Smoke.mpg". How would it look in the xml file? I hope I am not being too much of a weasel asking....I know I will spin my wheels later tonight when I try to get it working, so any tips would really be appreciated!

[stuckless]

Originally Posted by kmp14

Regex always makes my head hurt...

So for each show that I want BMT to try to do its' magic on, do I need to create an xml file for each, or do I add sections to the "master" SageRecordingAiring.xml?

Also, if possible, help me out with the regex. Say I have a file named "Mad Men--We Like To Smoke.mpg". How would it look in the xml file? I hope I am not being too much of a weasel asking....I know I will spin my wheels later tonight when I try to get it working, so any tips would really be appreciated!

Maybe evilpenguin will post his file scraper file (should only need one) once he's done... but if he doesn't, the ping me and I'll help you create a scraper file. It looks much harder than it really is

[kmp14]

Originally Posted by stuckless

Maybe evilpenguin will post his file scraper file (should only need one) once he's done... but if he doesn't, the ping me and I'll help you create a scraper file. It looks much harder than it really is

Thanks, I will give it a go tonight and beg for more help if needed!

[kmp14]

...but just to confirm, BMT will pick up any XML file in that folder and attempt to process it as a scraper? There can be multiple files, each of which represent a single scraper?

[stuckless]

Originally Posted by kmp14

...but just to confirm, BMT will pick up any XML file in that folder and attempt to process it as a scraper? There can be multiple files, each of which represent a single scraper?

You need one scraper file for each filename pattern that you want to match. ie, there is a scraper file for sage recordings; there is a scraper file for S##E##; etc.

So if your files were in the format "Series -- Episode", then you'd need one scraper file to process this filename pattern. You don't create a scraper for each Series name, just one scraper for the filename pattern itself. (make sense??)

[kmp14]

Complete sense. Thanks!

[kmp14]

No luck so far. In the folder \SageTV\STVs\Phoenix\scrapers\xbmc\tvfilenames there is a file named "Show-Episode.xml". Here are the contents of that file:

<?xml version="1.0" encoding="UTF-8"?>
<!-- 
   TV Filename Scraper.  Used to Scrape Show Name, AiringTitle, and AiringId from a Sage Recording
   $$1 is always the complete file uri
-->
<scraper name="title-sageRecording" content="filename" thumb="thumb.png">
	<!-- 	input: 		$1=complete file uri -->
	<!-- 	returns: 	series name -->
	<GetShowName dest="3">
		<RegExp input="$$1" output="\1" dest="3">
			<expression>.*[/\\]([^-]+)-([.*]+)\.</expression>
		</RegExp>
	</GetShowName>

	<!-- 	input: 		$1=complete file uri -->
	<!-- 	returns: 	Compressed Episode Title from the Sage Recording -->
	<GetEpisodeTitle dest="3">
		<RegExp input="$$1" output="\2" dest="3">
			<expression>.*[/\\]([^-]+)-([.*]+)\.</expression>
		</RegExp>
	</GetEpisodeTitle>
</scraper>

I renamed the files to be show-episode.mpg, but BMT did not find the metadata for them.

Do I need to move that "Show-Episode.xml" file up a folder? Is it not right? Looks right to me.

[stuckless]

Your scraper looks correct, but.... I have a bug in how the tv scrapers work, and I was not calling the GetEpisodeName function in the scraper except for a sage airing. I've fixed this and uploaded the 2.0.9 of phoenix-core.

So, with 2.0.9, your scraper should work, but be careful. That scraper is pretty generic, and it match any movie that has a dash in the filename as well. Given the tvscrapers are executed first, then then if you have a movie like...

Movie - 2010.avi, then it will detect that as a tv file with a series title of "Movie" and a episode name of 2010. That's why evilpenguin was talking about using a double dash in his tv filenames, so that it would not get confused with simply movie filenames.

ie, Show -- Episode.avi instead of Show - Epsisode.avi, and then account for the -- in the scraper.

[kmp14]

Hum, even after the 2.0.9 update, no luck. Here is one of the file names:

Mad Men -- The Rejected.mpg

Here is the scraper (which is in this folder: \SageTV\STVs\Phoenix\scrapers\xbmc\tvfilenames) :

<?xml version="1.0" encoding="UTF-8"?>
<!-- 
   TV Filename Scraper.  Used to Scrape Show Name, AiringTitle, and AiringId from a Sage Recording
   $$1 is always the complete file uri
-->
<scraper name="Get-Show-Episode" content="filename" thumb="thumb.png">
	<!-- 	input: 		$1=complete file uri -->
	<!-- 	returns: 	series name -->
	<GetShowName dest="3">
		<RegExp input="$$1" output="\1" dest="3">
			<expression>.*[/\\]([^-]+)--([.*]+)\.</expression>
		</RegExp>
	</GetShowName>

	<!-- 	input: 		$1=complete file uri -->
	<!-- 	returns: 	Compressed Episode Title from the Sage Recording -->
	<GetEpisodeTitle dest="3">
		<RegExp input="$$1" output="\2" dest="3">
			<expression>.*[/\\]([^-]+)--([.*]+)\.</expression>
		</RegExp>
	</GetEpisodeTitle>
</scraper>

The series and episode name are correct, and I can it on TheTVDB.com.

Here is the results from phoenix-metadata.log:

2010-08-17 21:16:17,828 - ERROR; T:\___TiVo\__TiVo\Mad Men -- The Rejected.mpg; Mad Men -- The Rejected; 2251231; Failed to get Metadata from the list of results

And the results in phoenix.log:

2010-08-17 21:16:17,513 [Thread-37] WARN  sagex.phoenix.metadata.search.MovieFileNameUtils - Failed to parse move title using scrapers for: T:\___TiVo\__TiVo\Mad Men -- The Rejected.mpg, will use the following movie title: Mad Men -- The Rejected
2010-08-17 21:16:17,514 [Thread-37] INFO  sagex.phoenix.metadata.MetadataManager - search(): imdb,tmdb; SearchQuery; Type: MOVIE; RAW_TITLE:Mad Men -- The Rejected;CLEAN_TITLE:Mad Men The Rejected;FILE:T:\___TiVo\__TiVo\Mad Men -- The Rejected.mpg;EPISODE_DATE:2010-08-16;
2010-08-17 21:16:17,515 [Thread-37] INFO  sagex.phoenix.metadata.MetadataManager - Searching: Mad Men -- The Rejected using IMDBMetadataProvider[MetadataProviderInfo [id=imdb, fanartId=tmdb, mediaTypes=[MOVIE], name=IMDb]]
2010-08-17 21:16:17,523 [Thread-37] INFO  sagex.phoenix.util.url.URLSaxParser - Parsing Url: http://www.imdb.com/find?s=tt&q=Mad+Men+--+The+Rejected&x=0&y=0
2010-08-17 21:16:17,523 [Thread-37] INFO  sagex.phoenix.util.url.CachedUrlFactory - Caching URL Factory in use.
2010-08-17 21:16:17,806 [Thread-37] INFO  sagex.phoenix.metadata.MetadataManager - Searching Using Cleaned Title: Mad Men The Rejected using IMDBMetadataProvider[MetadataProviderInfo [id=imdb, fanartId=tmdb, mediaTypes=[MOVIE], name=IMDb]]
2010-08-17 21:16:17,806 [Thread-37] INFO  sagex.phoenix.util.url.URLSaxParser - Parsing Url: http://www.imdb.com/find?s=tt&q=Mad+Men+The+Rejected&x=0&y=0
2010-08-17 21:16:17,822 [Thread-37] INFO  sagex.phoenix.metadata.MetadataManager - Searching: Mad Men The Rejected using TheMovieDBMetadataProvider[MetadataProviderInfo [id=tmdb, fanartId=tmdb, mediaTypes=[MOVIE], name=themoviedb.org]]
2010-08-17 21:16:17,825 [Thread-37] INFO  sagex.phoenix.metadata.provider.tmdb.TheMovieDBMetadataProvider - Fetching tmdb url: CachedUrl: http://api.themoviedb.org/2.1/Movie.search/en/xml/d4ad46ee51d364386b6cf3b580fb5d8c/Mad%2BMen%2BThe%2BRejected; UrlId: 02e28c59839e535d492d9161b6ac7963
2010-08-17 21:16:17,827 [Thread-37] WARN  sagex.phoenix.metadata.provider.tmdb.TheMovieDBMetadataProvider - TMDB Search for SearchQuery; Type: MOVIE; RAW_TITLE:Mad Men -- The Rejected;CLEAN_TITLE:Mad Men The Rejected;FILE:T:\___TiVo\__TiVo\Mad Men -- The Rejected.mpg;QUERY:Mad Men The Rejected;EPISODE_DATE:2010-08-16; returned no results.
2010-08-17 21:16:17,829 [Thread-37] WARN  phoenix.log - AutomaticMetadataVisitor(): lookup failed with an error for DecoratedItem: [SageMediaFile [sageId=2251231, sageObject=MediaFile[id=2251231 A[2251234,2251232,"__TiVo/Mad Men -- The Rejected",0@0816.00:24,63] mask=V host=Basement encodedBy= format=MPEG2-PS 1:03:59 2884 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80020001]] T:\___TiVo\__TiVo\Mad Men -- The Rejected.mpg, Seg0[Mon 8/16 0:24:54.502-Mon 8/16 1:28:54.279]]]]
MetadataException [query=SearchQuery; Type: MOVIE; RAW_TITLE:Mad Men -- The Rejected;CLEAN_TITLE:Mad Men The Rejected;FILE:T:\___TiVo\__TiVo\Mad Men -- The Rejected.mpg;QUERY:Mad Men The Rejected;EPISODE_DATE:2010-08-16;, ]
	at sagex.phoenix.metadata.MetadataManager.getMetdata(MetadataManager.java:476)
	at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:556)
	at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:526)
	at sagex.phoenix.metadata.AutomaticMetadataVisitor.visit(AutomaticMetadataVisitor.java:24)
	at sagex.phoenix.vfs.DecoratedMediaFile.accept(DecoratedMediaFile.java:48)
	at sagex.phoenix.vfs.DecoratedMediaFolder.accept(DecoratedMediaFolder.java:61)
	at sagex.phoenix.metadata.PhoenixMetadataSupport$1.run(PhoenixMetadataSupport.java:110)
	at sagex.phoenix.progress.ProgressTrackerManager$1.run(ProgressTrackerManager.java:48)

I am sure I am doing something wrong....just not sure what! I really appreciate the help!

[kmp14]

...I also tried the filename without the spaces before and after the dashes (Mad Men--The Rejected.mpg)

[evilpenguin]

I'm having some serious BMT related issues today so I haven't had a chance to try it out myself but that RegEx could be a bit problematic as it doesn't exclude slashes from the season/episode name, try this one as its a bit tighter...

.*[/\\]([^/\\-]+)-([^/\\]+)\.[a-zA-Z]+$

[kmp14]

Originally Posted by evilpenguin

I'm having some serious BMT related issues today so I haven't had a chance to try it out myself but that RegEx could be a bit problematic as it doesn't exclude slashes from the season/episode name, try this one as its a bit tighter...

.*[/\\]([^/\\-]+)-([^/\\]+)\.[a-zA-Z]+$

Awesome, Awesome, Awesome, it worked! Thanks so much. I gotta learn me some RegEx. I have run up against it a couple times and never just forced myself to learn it. Thanks again, you rock!

[stuckless]

Originally Posted by evilpenguin

I'm having some serious BMT related issues today

Anything you'd like to share? I don't like it when people have serious bmt issues.

[evilpenguin]

Well, not really BMT issues, but metadata issues in general. I'm still trying to sort what's my fault and what's with BMT, i'll let you know when I get to the bottom of it all