Utility: Batch Metadata Importer 3.x

[JREkiwi]

Sean,

I'm trying to get this set up and I can't figure out if I'm going to need to rename my imported videos, or if there is a configuration option I'm missing.

I'm trying to Web UI scan for fanart for imported videos with the filename format like this "Battlestar Galactica 4-01 He That Believeth in Me.mkv", but it seems to be doing a search for a movie called "Battlestar Galactica 4-01 He That Believeth in Me".

Is there a way to configure BMT/Phoenix to recognize this filename format?

Phoenix Version: 1.58, sagex.api version: 6.6.2, Metadata tools version: 3.1

John

[stuckless]

Originally Posted by JREkiwi

Sean,

I'm trying to get this set up and I can't figure out if I'm going to need to rename my imported videos, or if there is a configuration option I'm missing.

I'm trying to Web UI scan for fanart for imported videos with the filename format like this "Battlestar Galactica 4-01 He That Believeth in Me.mkv", but it seems to be doing a search for a movie called "Battlestar Galactica 4-01 He That Believeth in Me".

Is there a way to configure BMT/Phoenix to recognize this filename format?

Phoenix Version: 1.58, sagex.api version: 6.6.2, Metadata tools version: 3.1

John

John, take a look at scrapers/xbmc/tvfilenames/Title-s00e00.xml. You can either modify it to look for #-#. Or, if you don't want to modify it (because it will be overwritten on another install), just copy the file to another name and edit the new one. BMT loads all title scrapers in the tvfilenames directory.

[Islander]

I have been trying to find some specifics and searching found this:

48 Add MyMovies xml support
MyMovies support is added in the same way that DVDProfiler support is added. You provider BMT with the location of your MyMovies xml file and it index it and use it as a metadata source. BMT will re-index the xml file automatically as the file changes. MyMovies doesn't require you to set an images folder, since it stores the complete path to its images in its xml file. Set the following property in your metadata.properties file to identify the MyMovies xml file.

Code:
/mymovies/xmlFile=

If I understand the steps involved correctly:

1. Move MyMovies to the top of the list of providers. (I can do this)
2. Update metadata.properties. (I cannot find a specific file with this exact name unless it is the "sample" one with not much on it. What is the exact name and where is it located?)
3. I am using MyMovies WHS and I am not able to find one central xml file as I believe one for each title is stored in its own folder. Which sub do I link? The main share - top level?
4. I am happy with the meta already fetched by MM but I am done ripping my DVD collection and thus BMT already has metadata. How do I get it to replace the current meta with the MM one, of course after taking care of 1-3 above.

Thanks for your assistance.

[JREkiwi]

Originally Posted by stuckless

John, take a look at scrapers/xbmc/tvfilenames/Title-s00e00.xml. You can either modify it to look for #-#. Or, if you don't want to modify it (because it will be overwritten on another install), just copy the file to another name and edit the new one. BMT loads all title scrapers in the tvfilenames directory.

Thanks Sean, that's got it downloading the fanart. It was probably finger trouble, but it didn't download the fanart during the BMT scan, but downloaded it after I turned on the Automatic Plugin and did a SageTV media refresh.

I just need to figure out how to get the Phoenix to find the fanart, or is that something that's not configurable.

John

[JREkiwi]

It appears that updating the metadata gets it working with my filenames.

Have noticed one irritation though. Our EPG often has no episode name, but the episode description is correct. When the metadata gets updated, the show title is written into the episode field and the description is replaced with generic show information, so all the shows have the same description. Is there a way to stop it doing that?

John

[stuckless]

Originally Posted by JREkiwi

It appears that updating the metadata gets it working with my filenames.

Have noticed one irritation though. Our EPG often has no episode name, but the episode description is correct. When the metadata gets updated, the show title is written into the episode field and the description is replaced with generic show information, so all the shows have the same description. Is there a way to stop it doing that?

John

Not in 3.x.... In 4.0, I have added some options so that epg metadata is not overwritten (after several complaints, such as yours). My apologies for bmt overwriting your correct metadata. If bmt is only bringing down generic information, then that's a symptom that it's not actually finding valid episode data, but only generic series info data.

In 4.0, the default behaviour will be not NOT touch the wiz.bin for TV recordings.

[stuckless]

Originally Posted by Islander

I have been trying to find some specifics and searching found this:

48 Add MyMovies xml support
MyMovies support is added in the same way that DVDProfiler support is added. You provider BMT with the location of your MyMovies xml file and it index it and use it as a metadata source. BMT will re-index the xml file automatically as the file changes. MyMovies doesn't require you to set an images folder, since it stores the complete path to its images in its xml file. Set the following property in your metadata.properties file to identify the MyMovies xml file.

Code:
/mymovies/xmlFile=

If I understand the steps involved correctly:

1. Move MyMovies to the top of the list of providers. (I can do this)
2. Update metadata.properties. (I cannot find a specific file with this exact name unless it is the "sample" one with not much on it. What is the exact name and where is it located?)
3. I am using MyMovies WHS and I am not able to find one central xml file as I believe one for each title is stored in its own folder. Which sub do I link? The main share - top level?
4. I am happy with the meta already fetched by MM but I am done ripping my DVD collection and thus BMT already has metadata. How do I get it to replace the current meta with the MM one, of course after taking care of 1-3 above.

Thanks for your assistance.

The mymovies support that was added has never been tested by me... since I don't use/have mymovies, and I don't even run windows

But, my understanding is that you'll need a central xml collection file that you configure in the web ui (Configuration -> Metadata -> My Movies).

The MyMovies (and DVD Profiler) implementation works by indexing your entire collection xml and then when a movie is added to sage, it then searches your collection xml for the movie and pulls the metadata from it, including references to fanart, and writes it in a format that sage can use.

Another user has mentioned that they are using a different tool, but they they have individual xml files per title. In 4.0 of bmt (not released) i now have the facility to do lookups based on filename, so there is potential that I can better support the notion of individual xml files per title, but not until 4.0 is released. (ie, it would probably come as a 4.1 release).

as for the metadata.properties, you can ignore that for now... since it will be going away, and you only needed when you are trying to "force" a match on a series/movie where they are multiple matches returned, and bmt can't determine the best match.

[stuckless]

Originally Posted by JREkiwi

Thanks Sean, that's got it downloading the fanart. It was probably finger trouble, but it didn't download the fanart during the BMT scan, but downloaded it after I turned on the Automatic Plugin and did a SageTV media refresh.

I just need to figure out how to get the Phoenix to find the fanart, or is that something that's not configurable.

John

Phoenix automatically find the fanart based on 2 core metadata values. MediaType (TV/Movie) and MediaTitle. Both of these fields should be populated by bmt, but they require the the custom metadata fields property be setup. You can probably check this from the web ui (Configuration -> SageTV). In 3.x, it's hard to tell if those fields are being set, but in 4.0, i've added a "debug" screen that clearly shows the metadata values that's stored in the wiz.bin, properties, and custom metadata fields. Should make troubleshooting a little easier.

[JREkiwi]

Originally Posted by stuckless

Phoenix automatically find the fanart based on 2 core metadata values. MediaType (TV/Movie) and MediaTitle. Both of these fields should be populated by bmt, but they require the the custom metadata fields property be setup. You can probably check this from the web ui (Configuration -> SageTV).

Thanks Sean, It all started working once I had BMT update the metadata.

I've been meaning to update my Edit Metadata panel to add support for some of the values used by Phoenix. Looks like I need to at least have MediaType and MediaTitle included.

John

[stuckless]

Originally Posted by JREkiwi

Thanks Sean, It all started working once I had BMT update the metadata.

I've been meaning to update my Edit Metadata panel to add support for some of the values used by Phoenix. Looks like I need to at least have MediaType and MediaTitle included.

John

Yeah, those 2 are really the only important ones, especially for fanart.

[phelme]

Alright, I'm confused by the metadata-titles.properties file and how it works.

I'm trying to get fanart for "Survivor: Heroes vs. Villains" to show up. theTVDB has it listed as simply "Survivor". Based on the info in the thread, I placed this in my metadata-titles.properties file and restarted the SageTV server:

Survivor\:\ Heroes\ vs\.\ Villains=tvdb\:76733

but I'm not getting any fanart being downloaded and no (empty) directories are being created. Does this redirect create a "Survivor" folder where my fanart is and point the entry to it or does it create a "Survivor: Heroes vs. Villains" folder?

Same goes for "The Daily Show with Jon Stewart" which is only "The Daily Show" on theTVDB. Haven't been able to get that to work either.

I've tried manually doing a rescan through the web interface too. I must be missing something somewhere... maybe an entry in my Sage properites file?

does this work system-wide if it's on the server metadata-titles.properties file? or do I need to need to change the metadata-titles.properties file at each client (PC) as well?

I also notice there are already some entries in the metadata-titles.properties file that I didn't place there, what did??

thanks in advance.

[jryan15]

any chance that there is a way to clear the failed mediafiles list?

also is there anyway to manually update metadata? the 2009 movie "nine" is listed in imdb, but all of my bmt searches provide data for the 2009 movie "9" (returns nine for the title). thanks!

[stuckless]

Originally Posted by phelme

Alright, I'm confused by the metadata-titles.properties file and how it works.

I'm trying to get fanart for "Survivor: Heroes vs. Villains" to show up. theTVDB has it listed as simply "Survivor". Based on the info in the thread, I placed this in my metadata-titles.properties file and restarted the SageTV server:

Survivor\:\ Heroes\ vs\.\ Villains=tvdb\:76733

but I'm not getting any fanart being downloaded and no (empty) directories are being created. Does this redirect create a "Survivor" folder where my fanart is and point the entry to it or does it create a "Survivor: Heroes vs. Villains" folder?

Same goes for "The Daily Show with Jon Stewart" which is only "The Daily Show" on theTVDB. Haven't been able to get that to work either.

I've tried manually doing a rescan through the web interface too. I must be missing something somewhere... maybe an entry in my Sage properites file?

does this work system-wide if it's on the server metadata-titles.properties file? or do I need to need to change the metadata-titles.properties file at each client (PC) as well?

In 3.x the metadata.properties is used to pick a better match in a list results. ie, the search has to return a bunch of results and then the metadata.properties is used to pick a better results.

This system is flawed, since it requires that a search return results. If you have a search that doesn't return any results, or if your result is not in the list, then the metadata.properties is useless. Because of this, I've changed this in 4.0, so that instead of consulting the metadata.properties after the search, it actually will consult it before the search even starts. That allows you to setup a better search criteria before the search even happens.

I also notice there are already some entries in the metadata-titles.properties file that I didn't place there, what did??

Those get added when you use the on-demand searching from the STV.

Originally Posted by jryan15

any chance that there is a way to clear the failed mediafiles list?

Restarting sagetv should clear the list. (yeah I know, not very convenient)

also is there anyway to manually update metadata? the 2009 movie "nine" is listed in imdb, but all of my bmt searches provide data for the 2009 movie "9" (returns nine for the title). thanks!

If you browse the videos from the web ui, then you should be able to select the movie, and then manually edit the metadata.

[dkopp]

Stuckless, great addition to Sage.
I have a pile of recorded, downloaded and collected. Unfirtunalty the filenames are "show name - episode name". The scan does not pick up the metadata however if I manually set it as TV, cut and paste the show name and then episode name, BMI, always finds it and adds the metadata correctly.

I must be doing something wrong, if all the data needed (show and episode) are in the files name, where and how do I get BMI to correctly parse the name and then search??

Thank you

[stuckless]

Originally Posted by dkopp

Stuckless, great addition to Sage.
I have a pile of recorded, downloaded and collected. Unfirtunalty the filenames are "show name - episode name". The scan does not pick up the metadata however if I manually set it as TV, cut and paste the show name and then episode name, BMI, always finds it and adds the metadata correctly.

I must be doing something wrong, if all the data needed (show and episode) are in the files name, where and how do I get BMI to correctly parse the name and then search??

Thank you

You problem is most likely that BMT doesn not have a TV filename scraper for your format. BMT can lookup shows by episode title, which is why it works from the web ui.

You could add a tvfilename scrapert (scrapers/xbmc/tvfilenames/) but it requires some understanding of regex. If you look at the SageRecordingAiring.xml scraper you can see that it is used to parse out the Show and Episode from a filename, as long as the filename matches the sage recording format.

I'd recommend using the commandline for scraper testing...

java -jar MetadataTool.jar filename.avi

USing the commandline, you should see a .properties file created. You can then inspect the .properties to see if it's correctly grabing SeasonNumber and EpisodeNumber. Once you are happy with the scraper, then you can do a full scan (either using commandline or webui)

[phelme]

Originally Posted by stuckless

...

This system is flawed, since it requires that a search return results. If you have a search that doesn't return any results, or if your result is not in the list, then the metadata.properties is useless. Because of this, I've changed this in 4.0, so that instead of consulting the metadata.properties after the search, it actually will consult it before the search even starts. That allows you to setup a better search criteria before the search even happens.

Thanks Stuckless!!

[dkopp]

Originally Posted by stuckless

You problem is most likely that BMT doesn not have a TV filename scraper for your format. BMT can lookup shows by episode title, which is why it works from the web ui.

You could add a tvfilename scrapert (scrapers/xbmc/tvfilenames/) but it requires some understanding of regex. If you look at the SageRecordingAiring.xml scraper you can see that it is used to parse out the Show and Episode from a filename, as long as the filename matches the sage recording format.

I'd recommend using the commandline for scraper testing...

java -jar MetadataTool.jar filename.avi

USing the commandline, you should see a .properties file created. You can then inspect the .properties to see if it's correctly grabing SeasonNumber and EpisodeNumber. Once you are happy with the scraper, then you can do a full scan (either using commandline or webui)

would it be possible for someone to create the scraper for this format?

GaryUnmarried-GaryGivesSashaHisFullAttention or
Gary Unmarried - Gary Gives SashaHis Full Attention

I appreciate all the efforts of everyone. I am not a programmer but a pretty goog geek.

I have been thinking about creating a roadmap to the ideal media center. It would include BMT, TVExplorer, etc. So far these are my favorite plugins.

So.....I offer my help in merging the best of the best

[stuckless]

@dkopp, copy this and save it to Show-Episode.xml in the scrapers/xbmc/tvfilenames/ directory.

<?xml version="1.0" encoding="UTF-8"?>
<!-- 
   TV Filename Scraper.  Used to Scrape Show Name - Episode Title format
   $$1 is always the complete file uri
-->
<scraper name="Show-Episode" content="filename" thumb="thumb.png">
	<!-- 	input: 		$1=complete file uri -->
	<!-- 	returns: 	series name -->

	<GetShowName dest="3">
		<RegExp input="$$1" output="\1" dest="3">
			<expression>.*/([^-]+)-(.+)\.</expression>
		</RegExp>
	</GetShowName>

	<!-- 	input: 		$1=complete file uri -->
	<!-- 	returns: 	Compressed Episode Title from the Filename-->
	<GetEpisodeTitle dest="3">
		<RegExp input="$$1" output="\2" dest="3">
			<expression>.*/([^-]+)-(.+)\.</expression>
		</RegExp>
	</GetEpisodeTitle>
</scraper>

I would still do some command line testing before running it through a full scan.

[stuckless]

Just wanted to let people know that there is a new beta thread for discussions on the upcoming 4.0 release. (there is a beta download there as well)

We should leave this thread for 3.x discussions.

[bikesquid]

Originally Posted by stuckless

The first thing that I'd do, is see if those shows are actually defined in tvdb. If not, then that will account for the Failed. If they are, then email the log file (sagetv_0.txt) to me, and I do a quick check to see if there is another underlying issue.

Hi Sean,
Emailed you some log info. In the meantime saw the post regarding intro of 4.0 .

I'm wondering if switching to 4.0 would be a better way to start resolving my issues? I'm always reluctant to start changing things that aren't working yet.... Any suggestions?

Thx.
Ben.