Utility: Batch Metadata Importer 3.x

[Tiki]

Originally Posted by tmiranda

do you know if Tiki is altering his folder structure to comply with yours?

It is my intention to implement the Phoenix API's eventually because I think they are a good idea. However, I have limited time, I'm not the greatest programmer in the world, and things have been moving quickly. So, I have been waiting for things to stabilize before I take this on. My guess is that I will start experimenting in the next week or two and release something maybe in a month.

-Tom (TiKi)

[tmiranda]

Tiki,

OK, I't will be good when the "standards" settle out. Personally, I like your method better. It's simple yet effective. Having said that, I really do not care what "standard" is used as long as all of the tools (STV, STVi's, scrapers, etc.) comply with it.

Thanks for all of your hard work. I keep threatening to dust off my programming skills and learn Studio and Java. Unfortunately for me the last time I was a programmer I specialized in Unix device drivers and considered anything above the kernel as an "application"!

Tom

[LehighBri]

I recently downloaded this and installed and love it so far. One potential bug I noticed is that when I did my full reindex, it found my DVD fanart just fine, but it stored the picture files in the DVD folder rather than in my central fanart folder. Is that by design? Is there a workaround to store it in the FanArt\Movies folder similar to how TV is stored in FanArt\TV?

EDIT: Some more detail. I am using the default install, no metadata.properties file, default STV. Imported the BMT.stvi and central fanart folder set to T:\FanArt. Forcing reindex did not pull in the TV images to T:\FanArt\TV. The only way to get images to T:\FanArt\TV is to do the manual scan on a recording.

[stanger89]

Supid question, but is there a way to set the priority of provider usage? Eg can I set it to use DVD Profiler for everything, but to suppliment that with other sources (like fanart, etc)?

-er, that's probably what this line is...

/metadata/defaultProviderId

[onlydarksets]

Originally Posted by LehighBri

I recently downloaded this and installed and love it so far. One potential bug I noticed is that when I did my full reindex, it found my DVD fanart just fine, but it stored the picture files in the DVD folder rather than in my central fanart folder. Is that by design? Is there a workaround to store it in the FanArt\Movies folder similar to how TV is stored in FanArt\TV?

EDIT: Some more detail. I am using the default install, no metadata.properties file, default STV. Imported the BMT.stvi and central fanart folder set to T:\FanArt. Forcing reindex did not pull in the TV images to T:\FanArt\TV. The only way to get images to T:\FanArt\TV is to do the manual scan on a recording.

Ditto. I also tried this, but got an error:
java -jar MetadataTool.jar --fanartFolder "g:\media\Fanart" g:\Media\Movies

Batch MetaData Tools (2.0)
Failed while applying arg: fanartFolder (true) to method: setForceThumbnailOverwrite
Batch MetaData Tools (2.0)
Import/Update Movie MetaData from a MetaData Provider.
ERROR: argument type mismatch
java.lang.IllegalArgumentException: argument type mismatch
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.jdna.cmdline.CommandLine.applyToAnnotated(CommandLine.java:278)
at org.jdna.metadataupdater.MetadataUpdater.main(MetadataUpdater.java:89)

[stuckless]

Originally Posted by stanger89

Supid question, but is there a way to set the priority of provider usage? Eg can I set it to use DVD Profiler for everything, but to suppliment that with other sources (like fanart, etc)?

-er, that's probably what this line is...

/metadata/defaultProviderId

Yeah, /metadata/defaultProviderId, is actaully the complete list of providers that you want to use, and the order they they should be used.

ie, if you have,
/metadata/defaultProviderId=dvdprofiler,imdb.xml,tvdb.xml

Then dvdprofiler will be used first, and if there isn't a good match, then it will try the next provider, etc, until it finds a good match.

If you are are looking to do something like combine the results from multiple providers, then that's a "composite provider" and the configuration allows for that as well, although it's a little more complicated to setup. A composite provider allows you to search using one provider and suplement the details using both the search provider and a separate detail provider. ie, you could setup a composite provider that will use the imdb provider to search and then supplement the details using dvdprofiler. The "themoviedb.org-2" provider in the configuration is an example of a composite provider that uses imdb to search and supplements the details using themoviedb.

Just so you know, can now set the provider execution order using the stvi provided for the default stv and sagemc.

[babgvant]

Originally Posted by stanger89

Supid question, but is there a way to set the priority of provider usage? Eg can I set it to use DVD Profiler for everything, but to suppliment that with other sources (like fanart, etc)?

-er, that's probably what this line is...

/metadata/defaultProviderId

if memory serves, providers are used in the order they are entered in the prop file.

if a provider can't fulfill a query type (movies v. tv for e.g.) the next one is used, I don't remember what happens if a provider can handle a type but doesn't return results.

[stuckless]

Originally Posted by LehighBri

I recently downloaded this and installed and love it so far. One potential bug I noticed is that when I did my full reindex, it found my DVD fanart just fine, but it stored the picture files in the DVD folder rather than in my central fanart folder. Is that by design? Is there a workaround to store it in the FanArt\Movies folder similar to how TV is stored in FanArt\TV?

EDIT: Some more detail. I am using the default install, no metadata.properties file, default STV. Imported the BMT.stvi and central fanart folder set to T:\FanArt. Forcing reindex did not pull in the TV images to T:\FanArt\TV. The only way to get images to T:\FanArt\TV is to do the manual scan on a recording.

A couple of questions....
When you say that you did a full reindex, is tha via the commandline, or using the plugin?

If it was from the commandline, then may have introduced some confusion about adding the stv screen to set the central fanart directory. That screen set the phoenix/sagemc fanart property for the central fanart folder, but doesn not set the central fanart folder property in the metadata properties. (i'll add a bug) If you run it from the commandline, will need to pass --fanartFolder=YOUR_FOLDER. When the BMT provider runs inside sage, it can pick up the fanart folder location by checking to see if the phoenix/sagemc fanart folder is set, and use it, but in the commandline, it relies solely on it's own metadata.properties which probably doesn have the folder location set.

The behaviour that you talking about, ie dropping the images in the dvd folder dir, is what happens when central fanart is enabled, but a central fanart folder dir is not set.

[stuckless]

Originally Posted by onlydarksets

Ditto. I also tried this, but got an error:
java -jar MetadataTool.jar --fanartFolder "g:\media\Fanart" g:\Media\Movies

This is a bug.... bascially that property was removed from the configuration set, and it's complaining because it's trying to apply that property value to bmt but there's no place for it... (the simple answer).

You can either remove your metadata.properties, or edit it and remove the line that contains that string something like /xxxxx/setForceThumbnailOverwrite

[LehighBri]

Originally Posted by stuckless

A couple of questions....
When you say that you did a full reindex, is tha via the commandline, or using the plugin?

If it was from the commandline, then may have introduced some confusion about adding the stv screen to set the central fanart directory. That screen set the phoenix/sagemc fanart property for the central fanart folder, but doesn not set the central fanart folder property in the metadata properties. (i'll add a bug) If you run it from the commandline, will need to pass --fanartFolder=YOUR_FOLDER. When the BMT provider runs inside sage, it can pick up the fanart folder location by checking to see if the phoenix/sagemc fanart folder is set, and use it, but in the commandline, it relies solely on it's own metadata.properties which probably doesn have the folder location set.

The behaviour that you talking about, ie dropping the images in the dvd folder dir, is what happens when central fanart is enabled, but a central fanart folder dir is not set.

I didn't do from commandline. Per the first post in this thread, I updated my sage.properties force_full_content_reindex=true . Was I not supposed to do that? What is the best way to redo this and reindex and have it work as I'm expecting?

[stuckless]

Originally Posted by LehighBri

I didn't do from commandline. Per the first post in this thread, I updated my sage.properties force_full_content_reindex=true . Was I not supposed to do that? What is the best way to redo this and reindex and have it work as I'm expecting?

Then it is a bug... i'll look further into it... because whether you do a manual scan or reindex, it should simply put the fanart in the location that you specified. Now it will drop the poster image in the dvd folder, for backward compatibility with the default stv (I could add in an option to NOT do this), but it should still create entries in the fanart folder location for Backgrounds/ Posters/ etc for each movie.

[LehighBri]

Originally Posted by stuckless

Then it is a bug... i'll look further into it... because whether you do a manual scan or reindex, it should simply put the fanart in the location that you specified. Now it will drop the poster image in the dvd folder, for backward compatibility with the default stv (I could add in an option to NOT do this), but it should still create entries in the fanart folder location for Backgrounds/ Posters/ etc for each movie.

Thanks for looking in to this. To summarize, the issues I experience after setting the force reindex flag in my sage.properties are:
1. DVD move images are saved in the DVD source directly and NOT in the central Fanart folder
2. TV images are not saved period. The only way I can get TV images to save is to use the manual option by hitting Menu on each recording or media file.

[stuckless]

This is something that I should have stressed a little more in the release notes, specifically since it will have huge impact on whether or not TV fanart gets picked correctly after a scan.

As of the last beta build of sage, it has support for storing extra metadata fields. The scapers have been updated to pass those properties to sage, but Sage has to be configured to recieve those properties. You can do that in one of 2 ways. The first, is by using the stv plugin and enabling the metadata plugin. The second way is to manually edit your sage.properties and adding this line...

custom_metadata_properties=MediaProviderDataID;MediaTitle;MediaType;OriginalAirDate;EpisodeTitle;EpisodeNumber;SeasonNumber;DiscNumber;UserRating

Either way, you should check to see if you have that property in your sage.properties. If not, then add it, because it will simplify the TV fanart.

If a tv show is imported into sage with those properties set, then MediaTitle will be used as the fanart folder dir (ie, Bones, House, etc), no matter what the actual title is. Also the media type would have been set to TV and if available, there may have been season specific fanart. Without those properties, then the central fanart system ONLY relies on the display title, which may be something like, "Bones S01 E02 - Some Description". I'm also working on a "work around" for this as well, but those extra metadata fieds are very important when you importing standard videos as "TV shows".

[stuckless]

Originally Posted by LehighBri

Thanks for looking in to this. To summarize, the issues I experience after setting the force reindex flag in my sage.properties are:
1. DVD move images are saved in the DVD source directly and NOT in the central Fanart folder
2. TV images are not saved period. The only way I can get TV images to save is to use the manual option by hitting Menu on each recording or media file.

If you could send me (post it here or PM me) your sagetv_0.txt log file, then maybe I could see something specific in there as to why this is failing.

[LehighBri]

Originally Posted by stuckless

The first, is by using the stv plugin and enabling the metadata plugin. The second way is to manually edit your sage.properties and adding this line...

custom_metadata_properties=MediaProviderDataID;MediaTitle;MediaType;OriginalAirDate;EpisodeTitle;EpisodeNumber;SeasonNumber;DiscNumber;UserRating

I am using the stv plugin, and when I checked my sage.properties, that property is indeed there. I recently deleted all my fanart subfolders, stopped sage, updated the force reindex flag to true, and restarted sage and no TV shows were populated in the Fanart\TV central folder.

See attached for the log.

[onlydarksets]

Originally Posted by stuckless

This is a bug.... bascially that property was removed from the configuration set, and it's complaining because it's trying to apply that property value to bmt but there's no place for it... (the simple answer).

You can either remove your metadata.properties, or edit it and remove the line that contains that string something like /xxxxx/setForceThumbnailOverwrite

I don't have anything in my metadata.properties file. It only happens when I use the --fanartFolder parameter. If I don't use that parameter, it runs fine, but it dumps the fanart in the same folder as the movie, which, of course, doesn't work with the Phoenix APIs.

[stuckless]

Originally Posted by LehighBri

I am using the stv plugin, and when I checked my sage.properties, that property is indeed there. I recently deleted all my fanart subfolders, stopped sage, updated the force reindex flag to true, and restarted sage and no TV shows were populated in the Fanart\TV central folder.

See attached for the log.

Thank for the log, it helped alot... From it I could determine a few things.
1. I need to add 1080p to the list of words to ignore when doing title searches... many of your movies were skipped because it contained 1080p in the filename.
2. It think for the dvds/movies where it is not downloading fanart, it's because a properties file exists for those movies, and they are basically being skipped. I'm changing that in the next build.
3. When you mentioned tv shows, were you referring to existinf recorded shows or other import tv shows. BMT plugin does not fetch metadata for recorded shows. The manual way will, but not the plugin.

I will try to have a new build out in the next few days, I'm still doing some testing. For now, you can see if you have any properties lying around for the dvds... (or other vidoes) and remove the properties and then do a reindex.

[stuckless]

Originally Posted by onlydarksets

I don't have anything in my metadata.properties file. It only happens when I use the --fanartFolder parameter. If I don't use that parameter, it runs fine, but it dumps the fanart in the same folder as the movie, which, of course, doesn't work with the Phoenix APIs.

Sorry I should learn to read It's getting late... your problem is that the property needs to be set using an =

java -jar MetadataTool.jar --fanartFolder="g:\media\Fanart" g:\Media\Movies

[babgvant]

Originally Posted by onlydarksets

I don't have anything in my metadata.properties file. It only happens when I use the --fanartFolder parameter. If I don't use that parameter, it runs fine, but it dumps the fanart in the same folder as the movie, which, of course, doesn't work with the Phoenix APIs.

if you pipe the complete properties list into a text file (open it in wordpad to get w32 line breaks) it's much easier to manage.

java -jar metadata.jar --showproperties >> metadata.properties

then you can edit the values you want to change like ignored keywords and additional file types (m2ts for e.g.)

the new rev is really great, I was going to migrate my current fanart but after seeing the results from a few test cases I ended up deleting it letting BMT work it's magic. thanks sean!

[toricred]

Originally Posted by stuckless

3. When you mentioned tv shows, were you referring to existinf recorded shows or other import tv shows. BMT plugin does not fetch metadata for recorded shows. The manual way will, but not the plugin.

Will BMT plugin ever support recorded shows (at least the fanart) in an automatic way? I can get it to download the fanart with the manual process, but that's kind of a pain.