Utility: Batch Metadata Importer 3.x

[stuckless]

Originally Posted by LehighBri

Just saw this over on the release notes for the new 6.5.14 beta just released:



Does this present opportunities to improve this plugin? Or is this not related?

This will enhance the manual fanart lookups greatly. Currenlty the manual fanart/metadata lookup didn't pull down the the metadata because there was no real time way to update the media file with MediaType and MediaTitle, which are required for the proper use of fanart. Now, the manual fanart/metadata search will be able to download metadata and fanart.

This doesn't help or hinder the plugin at all since the plugin could already set the custom metadata fields.

I think where we'll see the most potential for this API call is in the on demand metadata fanart use.

[LehighBri]

I added this as an enhancement request for your tracking.

http://code.google.com/p/customsaget...s/detail?id=86

[ehfortin]

Hi,

I was reading the website and the PDF that has been put together by someone here (sorry, just don't remember the name of that person but... thanks for the good work) and I'm not sure of what I should expect.

The tool is working as a plugin under SageMC. However, it is not searching by itself. I can run it from the command line and it will grab the fanarts and update the metadata and I can also do it from SageMC by going into the "option" menu of a specific movie and then, it will work. But it's not running automatically behing the scene. Is that normal? My understanding of what I read is that I should be able to extract a DVD in a folder and fanart should appear at some point (assuming the files are under SageTV control and the directory rescan is activated) but I was not sure of that.

Now, most of my recording are actual movies. The lookup for Recorded TV seems to be limited to TV metadata and fanart. I'm fine with the metadata part as SageTV is having it but... I would like to get the fanarts. I tried to run manually the tool with the TV stuff as if it was movie. It actually get FanArts but... it put it into Movies (which is expected) so they don't appear in SageTV.

Is there anyway to have the tool get the fanarts from the movies databases but that would appear in the "TV" directory (central fanart directory)? Is that what is not there and may come at some point?

That's it for now. It's not that late but... I seems like I'm not very clear. Let me know if you need some precision.

Thank you.


ehfortin

[sflamm]

doc updated.

[stuckless]

ehfortin, make sure that you have correctly configured the automatic plugin (Section 5) in sflamm's doc.

Also, the automatic plugin, is only as automatic as sage is, in checking for newly added media. So, after you add your media, you may need to tell Sage to refresh/rescan for new media, and this should trigger the plugin. If the plugin can't find a good match for a given media item, then it will remain as is, without metadata/fanart.

Also, if you check your sagetv_0.txt log file, you'll see some "BMT" messages, and it will tell you if it was able to find metadata or not.

[jaminben]

Originally Posted by stuckless

This will enhance the manual fanart lookups greatly. Currenlty the manual fanart/metadata lookup didn't pull down the the metadata because there was no real time way to update the media file with MediaType and MediaTitle, which are required for the proper use of fanart. Now, the manual fanart/metadata search will be able to download metadata and fanart.

This doesn't help or hinder the plugin at all since the plugin could already set the custom metadata fields.

I think where we'll see the most potential for this API call is in the on demand metadata fanart use.

Sweet, this is what I've been waiting for

[tmiranda]

Originally Posted by stuckless

tom, My "todo" list the the issue tracker for the most part... so thanks for adding an issue there. That way I won't forget about it

I'm currently working on better recorded tv support. In your case, are you referring to recorded tv shows that are in the wiz.bin (ie already known recordings) or are you referring to recorded tv shows that have been lost from the wiz.bin (ie does sage already have some metadata for the show, such as series name, episode title, etc)

And, are you referring to recorded shows via the automatic plugin or the commandline or both?

Sean,

I'm referring to recorded show that ARE in the wiz.bin. I'd like it to work via the automated plugin but if I have to occasionally run the tool manually that's OK too.

In my testing I have been using the manual command line method. I have not attempted the automated plugin at all.

Thanks,

Tom

[ehfortin]

Hi,

Stuckless, I've followed the instruction provided by sflamm. I even recheck the whole thing this morning. The line custom_metadata_properties=... is there. The mediafile_metadata_parser_plugin=org.jdna.sage.MetadataUpdaterPlugin is also there.

I just saw that there is a line called "phoenix/mediametadata/parseMediaFileForTVSeries=false". Should it be changed to "true"? Same thing for phoenix/mediametadata/fanartEnabled. It was originally set as false. I've changed this one as true but it doesn't seems to do much.

BTW, I don't seems to have the log file you are referring to. Is it supposed to be there by default or I need to set a switch?

As for the question regarding movies into the TV sections, I've reread the latest version of sflamm's doc and I'm still not sure if I understand correctly. It seems that it will trig the fanart but without changing the metadata (I'm fine with that) and will save these fanart in the TV subdirectory. When I run the tool manually, it is not finding any movie directly as the filename are allwithoutspace_stvid so I always have to confirm title from the title list. But as the movie recorded in SageTV have the correct name in the metadata, I would expect metadatatool to take the exact movie title from SageTV to do the lookup.

Is that how it should work? Or that is not the case yet?

Thank you.

ehfortin

[stuckless]

Originally Posted by tmiranda

Sean,

I'm referring to recorded show that ARE in the wiz.bin. I'd like it to work via the automated plugin but if I have to occasionally run the tool manually that's OK too.

In my testing I have been using the manual command line method. I have not attempted the automated plugin at all.

Thanks,

Tom

I think the 2.3 release is what you are after.... and no, I don't know when, but probably in a couple weeks. The TV support in the 2.3 release will support working with sage recorded media files, whether they are in the actual sage recordings or not.

BUT, just before anyone asks.... no the 2.3 release will NOT importing media into the sage recordings... that's probably a 2.4 feature at best.

[jaminben]

Will a 2.2.5 pre-release of 2.3 have support for on demand metadata/fanart Sorry I couldn't resist asking

[stuckless]

Originally Posted by jaminben

Will a 2.2.5 pre-release of 2.3 have support for on demand metadata/fanart Sorry I couldn't resist asking

Yeah... I'll make a 2.2.5-beta1 available... probably this weekend... I have it working now... but I need more testing.

[jaminben]

Originally Posted by stuckless

Yeah... I'll make a 2.2.5-beta1 available... probably this weekend... I have it working now... but I need more testing.

Nice one, I was only joking and didn't expect a beta so soon. I can always use EP's scraper to test my code but if you have a beta coming that'll make things easier

[sflamm]

Questions:

1. Are we suppose to be able to get background art for Videos? for Music?

I've been using the composite DVD Profiler + TheMovieDB provider and I dont seem to get any... I change to another provider and I seem to get a background, but no poster art... mixed results... not sure how to improve. Thoughts?

2. Has anyone noticed that in the FanArt sources UI, there appears to be a bug with the buttons at the bottom of the screen (occurs when you go back to the screen after the first time)

Picture of UI bug attached.

3. There is another bug with the FanArt sources UI where the buttons do not show up at all

Picture of 2nd UI bug attached.

4. My metadata.properties file has:

#Configuration Properties
#Fri Apr 24 22:07:29 PDT 2009
/compositeMetadataProviders/dvdprof2/compositeMode=1
/compositeMetadataProviders/dvdprof2/description=DVD Profiler, but get fanart and missing details from TheMovieDB
/compositeMetadataProviders/dvdprof2/detailProviderId=themoviedb.org
/compositeMetadataProviders/dvdprof2/iconUrl=
/compositeMetadataProviders/dvdprof2/name=DVD Profiler + TheMovieDB
/compositeMetadataProviders/dvdprof2/searchProviderId=dvdprofiler
/dvdprofilerLocal/imageDir=c\:\\SageTV\\DVD Profiler\\Databases\\Default\\Images
/dvdprofilerLocal/xmlFile=c\:\\SageTV\\DVD Profiler\\Databases\\Collection.xml
/metadata/airingIdRegex=([0-9]+)-[0-9]{1,2}\\.
/metadata/backgroundImageWidth=-1
/metadata/bannerImageWidth=-1
/metadata/defaultProviderId=dvdprofiler,tvdb,themoviedb.org,themoviedb.org-2,imdb,imdb.xml
/metadata/enableDefaultSTVPosterCompatibility=false
/metadata/goodScoreThreshold=0.9
/metadata/maxDownloadableImages=5
/metadata/persistenceClass=org.jdna.media.metadata.impl.sage.SageTVPropertiesWithCentralFanartPersistence
/metadata/posterImageWidth=200
/metadata/scoreAlternateTitles=true
/metadata/videoMetadataProviders=org.jdna.media.metadata.impl.tvdb.TVDBMetadataProvider,org.jdna.media.metadata.impl.imdb.IMDBMetaDataProvider,org.jdna.media.metadata.impl.nielm.NielmIMDBMetaDataProvider,,org.jdna.media.metadata.impl.dvdproflocal.LocalDVDProfMetaDataProvider,org.jdna.media.metadata.impl.themoviedb.TheMovieDBMetadataProvider,
/metadata/wordsToClean=1080p,720p,480p,1080i,720i,480i,dvd,dvdrip,cam,ts,tc,scr,screener,dvdscr,xvid,divx,avi,vrs,repack,mallat,proper,dmt,dmd,stv,HDTV,x264

but the log file produces:

2009-04-24 22:44:51,814  INFO [AWT-EventQueue-0] org.jdna.util.LoggerConfiguration (LoggerConfiguration.java:27) - Using default loggers from: C:\Program Files\SageTV\SageTV\log4j.properties
2009-04-24 22:44:51,814  INFO [AWT-EventQueue-0] bmt.api (api.java:25) - Metadata Tools API: 2.2; Using Phoenix: 1.22
2009-04-24 22:44:51,814  INFO [AWT-EventQueue-0] bmt.BMTMetadataSupport (BMTMetadataSupport.java:47) - Using BMTMetadataSupport: 2.2
2009-04-24 22:46:02,517  INFO [AWT-EventQueue-0] org.jdna.util.LoggerConfiguration (LoggerConfiguration.java:27) - Using default loggers from: C:\Program Files\SageTV\SageTV\log4j.properties
2009-04-24 22:46:02,517  INFO [AWT-EventQueue-0] bmt.api (api.java:25) - Metadata Tools API: 2.2; Using Phoenix: 1.22
2009-04-24 22:46:02,517  INFO [AWT-EventQueue-0] bmt.BMTMetadataSupport (BMTMetadataSupport.java:47) - Using BMTMetadataSupport: 2.2
2009-04-24 22:46:37,126  INFO [LibraryImporter] org.jdna.util.LoggerConfiguration (LoggerConfiguration.java:27) - Using default loggers from: C:\Program Files\SageTV\SageTV\log4j.properties
2009-04-24 22:46:37,142 DEBUG [LibraryImporter] org.jdna.configuration.ConfigurationManager (ConfigurationManager.java:86) - Loading: C:\Program Files\SageTV\SageTV\metadata.properties
2009-04-24 22:46:37,142 DEBUG [LibraryImporter] org.jdna.configuration.ConfigurationManager (ConfigurationManager.java:102) - Loading: C:\Program Files\SageTV\SageTV\metadata-titles.properties
2009-04-24 22:46:37,142  INFO [LibraryImporter] org.jdna.configuration.ConfigurationManager (ConfigurationManager.java:110) - No titles map configured: C:\Program Files\SageTV\SageTV\metadata-titles.properties
2009-04-24 22:46:37,189  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: thetvdb.com
2009-04-24 22:46:37,189  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: IMDb
2009-04-24 22:46:37,189  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: IMDb (Nielm)
2009-04-24 22:46:37,189  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: DVD Profiler
2009-04-24 22:46:37,189  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: themoviedb.org
2009-04-24 22:46:37,189 DEBUG [LibraryImporter] org.jdna.media.metadata.impl.composite.CompositeMetadataProvider (CompositeMetadataProvider.java:38) - Composite Provider Created with id: themoviedb.org-2; search: imdb; details: themoviedb.org; mode: 1
2009-04-24 22:46:37,189  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: IMDb/themoviedb.org
2009-04-24 22:46:37,220 DEBUG [LibraryImporter] org.jdna.media.metadata.impl.xbmc.XbmcMetadataProvider (XbmcMetadataProvider.java:83) - Provider:  imdb.xml; content type: movies
2009-04-24 22:46:37,220 DEBUG [LibraryImporter] org.jdna.media.metadata.impl.xbmc.XbmcMetadataProvider (XbmcMetadataProvider.java:85) - Using Movies for Provider:  imdb.xml; content type: movies
2009-04-24 22:46:37,220  INFO [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:109) - Adding MetaDataProvider: IMDb (Xbmc)
2009-04-24 22:46:37,236 DEBUG [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:136) - Multiple Provider Ids were passed; dvdprofiler,tvdb,themoviedb.org,themoviedb.org-2,imdb,imdb.xml; Creating a MetadataProviderContainer for them...
2009-04-24 22:46:37,236 DEBUG [LibraryImporter] org.jdna.media.metadata.MediaMetadataFactory (MediaMetadataFactory.java:136) - Multiple Provider Ids were passed; dvdprofiler,tvdb,themoviedb.org,themoviedb.org-2,imdb,imdb.xml; Creating a MetadataProviderContainer for them...
2009-04-24 22:46:37,236 DEBUG [LibraryImporter] org.jdna.media.impl.MovieResourceFilter (MovieResourceFilter.java:28) - Using Movie Filter Regex: avi|mpg|divx|mkv|wmv|mov|xvid|ts|m2ts|m4v|mp4|iso
2009-04-24 22:46:37,236 DEBUG [LibraryImporter] org.jdna.media.impl.MovieResourceFilter (MovieResourceFilter.java:32) - Not Using any Directory Exclude Filters.
2009-04-24 22:46:37,251 DEBUG [LibraryImporter] org.jdna.media.util.FileNameUtils (FileNameUtils.java:28) - Loaded Filename Scraper: title-s00d00
2009-04-24 22:46:37,251 DEBUG [LibraryImporter] org.jdna.media.util.FileNameUtils (FileNameUtils.java:28) - Loaded Filename Scraper: title-s00e00
2009-04-24 22:46:37,251 DEBUG [LibraryImporter] org.jdna.media.util.FileNameUtils (FileNameUtils.java:38) - Using Scrapers to find a query for: file:/C:/SageTV/Videos/Capote/
2009-04-24 22:46:37,251 DEBUG [LibraryImporter] org.jdna.persistence.PropertiesPersistence (PropertiesPersistence.java:167) - Attempting to load a map for: /xbmcScraper/Title-s00d00.xml/settings
2009-04-24 22:46:37,267 DEBUG [LibraryImporter] org.jdna.media.metadata.impl.xbmc.XbmcScraperProcessor (XbmcScraperProcessor.java:46) - XbmcScraperProcessor created using Scraper: title-s00d00; Content: filename; Complete Logging: false

It looks like it built the wrong composite provider??

Entire log file is attached.

[MeInMaui]

I'm planning to rewrite the UI and incorporate it into SageMC. I just need to get these other bugs worked out first.

Aloha,m
Mike

[tmiranda]

Originally Posted by stuckless

I think the 2.3 release is what you are after.... and no, I don't know when, but probably in a couple weeks. The TV support in the 2.3 release will support working with sage recorded media files, whether they are in the actual sage recordings or not.

OK, but I am confused. The documentation says that if the shows use the standard Sage naming format (series-episode-airingid-0.mpg) the tool will work. Am I doing something wrong or have I misread the documentation?

[stuckless]

Originally Posted by tmiranda

OK, but I am confused. The documentation says that if the shows use the standard Sage naming format (series-episode-airingid-0.mpg) the tool will work. Am I doing something wrong or have I misread the documentation?

This was started in the 2.1 release but won't be complete until the 2.3 release. In the 2.1 release this support was added to the Automatic Plugin and NOT the command line. If the automatic plugin detected a sage recording on a rescan, then it would try to download the fanart based on the airing id in the filename. Again, only for the plugin, and it would only fetch "series" level fanart.

In the 2.3 release, bmt will work with sage recordings in both the commandline and the automatic plugin. It will will also support fetching episode specific specific metadata and season specific fanart. From the commandline, it will attempt the use the remote apis, but if they are not installed, then it will still work, and it will use the series and episode title parsed from the sage recording filename. The series search and episode search will find a match even though the sage information in the recording is "munged" together.

So, you can see that in 2.1... it was started, (probably shouldn't even have mentioned it), but it really won't be functional until the next release.

[PLUCKYHD]

Originally Posted by jaminben

Will a 2.2.5 pre-release of 2.3 have support for on demand metadata/fanart Sorry I couldn't resist asking

Could you explain what you mean by this?

[tmiranda]

Originally Posted by stuckless

This was started in the 2.1 release but won't be complete until the 2.3 release. In the 2.1 release this support was added to the Automatic Plugin and NOT the command line. If the automatic plugin detected a sage recording on a rescan, then it would try to download the fanart based on the airing id in the filename. Again, only for the plugin, and it would only fetch "series" level fanart.

Gotcha. So if I install the plugin version it WILL download the fanart. (I'm OK with "series" level fanart.) This is great news! I will give it a try as soon as time permits.

Edit: The STVi for the default STV does not seem to work properly. I get the new "Metadata / Fanart" option on the Detailed Setup menu and when I select it I can see the two options "Fanart Enabled" and "Automatic Metadata/Fanart" but the buttons to the right of those options are blank. Clicking on them has no effect, no text is visible at all. I tried uninstalling the STVi and reimporting it several times with the same result.

The task manager shows the Metadata-updater-2.2 is running, but I have no idea what it is doing

If I want to uninstall this manually (I don't like the idea of the metadata-updater-2.2 process running) how do I do it? Is it safe just to delete all of the references to "metadata" in the sage.properties file?

[stuckless]

Originally Posted by tmiranda

Gotcha. So if I install the plugin version it WILL download the fanart. (I'm OK with "series" level fanart.) This is great news! I will give it a try as soon as time permits.

Edit: The STVi for the default STV does not seem to work properly. I get the new "Metadata / Fanart" option on the Detailed Setup menu and when I select it I can see the two options "Fanart Enabled" and "Automatic Metadata/Fanart" but the buttons to the right of those options are blank. Clicking on them has no effect, no text is visible at all. I tried uninstalling the STVi and reimporting it several times with the same result.

That's odd.. I'll take a look at that.

The task manager shows the Metadata-updater-2.2 is running, but I have no idea what it is doing

If I want to uninstall this manually (I don't like the idea of the metadata-updater-2.2 process running) how do I do it? Is it safe just to delete all of the references to "metadata" in the sage.properties file?

The metadata-updater process is not from the sage plugin. It's most likely from a failed commandline that didn't finish. You can probably just terminate the process.

When the metadata tools are running as a sagetv plugin, it runs in sage process and does not create a new process. ie, it runs like all other sage plugins, like the web server, jetty server, etc.

But if you do want to uninstall the plugin, just remove the following line from the Sage.properties

mediafile_metadata_parser_plugins=org.jdna.sage.MetadataUpdaterPlugin

[tmiranda]

I've never run the metadata-updater on my Sage server. I've been doing the command line testing on another machine.

The STVi is doing something strange.... Let me know if you need help debugging it.