Utility: Batch Metadata Importer 3.x

[stuckless]

Originally Posted by wayner

Thanks Sean - I was wondering about the .my files since that is a setting unique to SageMC and for EvilPenguin's MediaScraper there is a setting to generate properties files or My files which is for: "/genMyFile - Generate a .my file, contains extra data for SageMC "

I may look at adding support for .my files, once I complete all other outstanding tasks But I'm not sure what a .my file provides that would not be in a .properties file.

I saw something in your list of issues about deleting properties files - do you not need the properties files - that is, can they be deleted after being read by Sage? Or are they read every time you open up a show/movie?

.properties is the external way to get metadata into the SageTV wiz.bin file. Once metadata has been imported, the .properites file is no longer needed. That being said.... if your wiz.bin were to get corrupt, and you had to re-scan your collection, then having the .properties files would act as a backup for the metadata, and make rescanning a much faster operation. So, Once metadata has been imported, then the .properties are never read again unless you re-scan your collection. This "enhancement" is basically an option that would delete the file once the data was passed to SageTV from the automatic metadata plugin.

One further question - occasionally I am getting 0 byte jpg files for Fanart - have you seen this?

I have not see this, but I can certainly see how it could happen. I think I added an issue for this as well, if not, then I'll add an issue.

[VikingCrown]

I am also getting 0 byte fan art. I thought it was because the file i tried last night was recording but i tried it again this morning using the plugin and still a 0 byte image. Using: 6.3.9-41, SageBeta 6.5.15, and latest bmt, all attempts from plugin.

[stuckless]

0 byte fanart is most likely caused because the image stream could not be located during the write. Not sure what would cause this exactly, but I've opened a bug, so I'll investigate.

[electronics4lif]

Originally Posted by stuckless

If you disable fanart, then bmt behaves as it did before. ie, posters are written as "vidoe.jpg" in the same folder as the video item. I'll add a bug to take a look at this, but I currently use both scenarios in my home environment, and I have not noticed the issue that you have described.


On a side note... In a few days (or on the weekend) I'll be releasing a beta of the 3.0 release. I would encourage anyone that is currently "struggling" with the 2.x release to try the beta since you'd have nothing to lose anyways. The new beta will have web based configuration section that should make configuring and troubleshooting issues much more intuitive. In many cases, the issues that are reported, can never be reproduced, and I suspect that in some cases, it may simply be configuration related. Hopefully the new build will help with that by allowing a more visual approach to the configuration. The web configuration will depend on the Jetty webserver plugin, since will deploy a web configuration package to the Jetty plugin.

What is strange is that when I turn off fanart, the metadata is being retrieved but not the poster. I would think it would be all or nothing. I just tried it again to verify that I wasn't losing my mind. ) For now, I am turning fanart on but deleting the "backgrounds" folder from each movie. I look forward to your 3.0 release!

[Farm]

I'm in the process of setting up BMT on the server for my HD200 and a separate computer using the SageTV client.
So:
WHS Server running SageTV for WHS
Desktop running SageTV Client

I'm running the latest beta of SageMC 6.3.9a_41 and I've followed the installation instructions by sflamm.

If I understand the installation instructions for BMT I need to install BMT on both the server and the local client. I did that.

Then I added this to my metadata.properties file:
/dvdprofilerLocal/imageDir=\\\\D\\My Documents\\DVD Profiler\\Databases\\Default\Images
/dvdprofilerLocal/xmlFile=\\\\D\\My Documents\\DVD Profiler\\Databases\\Collection.xml
/compositeMetadataProviders/dvdprof2/name=DVD Profiler + TheMovieDB
/compositeMetadataProviders/dvdprof2/description=DVD Profiler, but get fanart and missing details from TheMovieDB
/compositeMetadataProviders/dvdprof2/iconUrl=
/compositeMetadataProviders/dvdprof2/searchProviderId=dvdprofiler
/compositeMetadataProviders/dvdprof2/detailProviderId=themoviedb.org
/compositeMetadataProviders/dvdprof2/compositeMode=1

I'm not sure if the lines in bold are correct. The path is correct, but is my formatting correct?

I'm unable to point my "Central Fanart Folder" to one of the WHS shares. The highest directory structure I can browse within SageTV is the local computer (C: D: F: etc.). It would be nice to have a central Fanart folder for all installations...is it possible to use one of the WHS shares for this?

When I try up update Metadata within SageTV it doesn't pull anything in from the DVDProfiler collection. It says something like "not found in the DB" if I try a DVDProfiler Refresh. I've tried renaming the movie to exactly what DVDProfiler calls it and several other variations as well.

I think I'm really close to getting this all to work, but I'm getting hung up on these final steps to actually import the DVDProfiler data and getting Fanart to work.

Any help is greatly appreciated.

[razrsharpe]

Originally Posted by Farm

/dvdprofilerLocal/imageDir=\\\\D\\My Documents\\DVD Profiler\\Databases\\Default\Images
/dvdprofilerLocal/xmlFile=\\\\D\\My Documents\\DVD Profiler\\Databases\\Collection.xml
/compositeMetadataProviders/dvdprof2/name=DVD Profiler + TheMovieDB
/compositeMetadataProviders/dvdprof2/description=DVD Profiler, but get fanart and missing details from TheMovieDB
/compositeMetadataProviders/dvdprof2/iconUrl=
/compositeMetadataProviders/dvdprof2/searchProviderId=dvdprofiler
/compositeMetadataProviders/dvdprof2/detailProviderId=themoviedb.org
/compositeMetadataProviders/dvdprof2/compositeMode=1

I'm not sure if the lines in bold are correct. The path is correct, but is my formatting correct?

I think those lines should read:

/dvdprofilerLocal/imageDir=D\:\\My Documents\\DVD Profiler\\Databases\\Default\\Images
/dvdprofilerLocal/xmlFile=D\:\\My Documents\\DVD Profiler\\Databases\\Collection.xml

Basically you need a \ in front of all special characters (: \ etc)

I'm unable to point my "Central Fanart Folder" to one of the WHS shares. The highest directory structure I can browse within SageTV is the local computer (C: D: F: etc.). It would be nice to have a central Fanart folder for all installations...is it possible to use one of the WHS shares for this?

yes you can click specify from that same dialog box then manually enter the path. You can also edit the .properties file directly with a text editor... if you go this route make sure sage is fully closed (client, ui, service all off)... and be careful with the slashies like above. You will need to do this on all clients.

When I try up update Metadata within SageTV it doesn't pull anything in from the DVDProfiler collection. It says something like "not found in the DB" if I try a DVDProfiler Refresh. I've tried renaming the movie to exactly what DVDProfiler calls it and several other variations as well.

with BMT you do not need to (and should not) use the sagemc dvd profiler support... im pretty sure sflamms install document says so... if you want to use the built in one (not recommended) there are two other options that you need to set... i think its in sagemc settting -> paths -> metadataXML and dvd profiler path. again if using bmt to pull the dvd profiler info you should not use the sagemc default dvdprofiler support.

Goodluck

[Farm]

Thanks for the advice.
I'll work on it in the morning and post the results.

[Farm]

OK, I changed the paths in the metadata.properties file to centralized folders:

/dvdprofilerLocal/imageDir=\\\\Server\\SageTV\\DVD Profiler\\Databases\\Default\\Images
/dvdprofilerLocal/xmlFile=\\\\Server\\SageTV\\DVD Profiler\\Databases\\Default\\Collection.xml

I made all the changes to disabling DVD profiler support built into SageMC.

Nothing updates automatically, even after a refresh. The only way I can get Fanart to update is to fo into each movie and select Metadata\Fanart.

Also when I do that, if the videos are sorted by "All Videos" then I see the cover art and a background fanart as well. IF I sort the videos by "Genre", the background fanart will show up as I scroll, but I have NO cover art for the videos. ????? Any ideas on that?

[Farm]

I'm not sure if this matters, but when I go to my

\\Server\SageTV\DVD Profiler\Databases\Default\Images

Folder, the thumbnails are all name crazy things. For Example, Monsters Inc. front cover jpg is called "786936164886f.jpg".

Do the jpg names in the DVDProfiler Images folder matter??

[matterw]

Running MetadataUpdater from the commandline and getting the error "SageAPI Provider is not set, will try to find the server...".

Not sure why it's trying to connect to the server, but how do I fix this exception being thrown?

Thanks,
Matt

[stuckless]

@Farm - The image name is not important. That is most likely the DVD Profiler imagename from your DVD Profiler images directory. DVD Profiler names their images as ########f.jpg and #########b.jpg for front and back respectively. BMT only uses the front images.

@matterw - You can download the latest Sage Remote apis, which has fix that should prevent it from trying to contact the server. Phoenix relies on the sagex apis, which for the most part is not an issues, since phoenix usually runs in the same process as the SageTV. BMT relies on phoenix for much of the fanart stuff, and as a result, the commandline will try to contact the server when doing some operations. The latest sagex apis has a temporary fix for this... basically it will still try and contact the server, but it should do it only once, and then fall back to a local implemenation for the apis. The upcomming BMT release has a more complete fix.

[Farm]

Thanks stuckless.

BMT still won't automate any metadat/fanart import. I'm not sure where I may have my signals crossed.

One differentiation I found in reading this thread is in the installation instruction by sflamm he states:

8.

Turn off “DVD Profiler” Meta Data refresh:
SageMC ->Paths ->Meta Data Xml = \\NAS\SageTV\DVD Profiler\Collection.xml
SageMC ->Paths ->Dvd Profiler Path = \\NAS\SageTV\DVD Profiler\Databases\Default
SageMC -> Main -> Screen Specific Options ->Video Library ->DVDProfiler Auto Meta Data "disabled"

But in post #663 on page 34 of this thread he states:

e. DVD profiler settings

meta data xml = \\NAS\SageTV\DVD Profiler\Collection.xml
dvd profiler path = \\NAS\SageTV\DVD Profiler\Databases\Default
Auto Load DVD Profiler Meta Data "enabled"

Here is my metadata.properties file:

#Configuration Properties
#Fri Jun 05 21:44:46 CDT 2009
/compositeMetadataProviders/dvdprof2/compositeMode=1
/compositeMetadataProviders/dvdprof2/description=DVD Profiler, but get fanart and missing details from TheMovieDB
/compositeMetadataProviders/dvdprof2/detailProviderId=themoviedb.org
/compositeMetadataProviders/dvdprof2/iconUrl=
/compositeMetadataProviders/dvdprof2/name=DVD Profiler + TheMovieDB
/compositeMetadataProviders/dvdprof2/searchProviderId=dvdprofiler
/dvdprofilerLocal/imageDir=\\\\Server\\SageTV\\DVD Profiler\\Databases\\Default\\Images
/dvdprofilerLocal/xmlFile=\\\\Server\\SageTV\\DVD Profiler\\Databases\\Default\\Collection.xml
/metadata/backgroundImageWidth=-1
/metadata/bannerImageWidth=-1
/metadata/defaultProviderId=dvdprofiler-2,tvdb,themoviedb.org,imdb.xml,imdb
/metadata/enableDefaultSTVPosterCompatibility=false
/metadata/goodScoreThreshold=0.9
/metadata/importTVAsRecordedShows=false
/metadata/maxDownloadableImages=5
/metadata/persistenceClass=org.jdna.media.metadata.impl.sage.SageTVPropertiesWithCentralFanartPersistence
/metadata/posterImageWidth=200
/metadata/scoreAlternateTitles=true
/metadata/videoMetadataProviders=org.jdna.media.metadata.impl.tvdb.TVDBMetadataProvider,org.jdna.media.metadata.impl.imdb.IMDBMetaDataProvider,org.jdna.media.metadata.impl.nielm.NielmIMDBMetaDataProvider,,org.jdna.media.metadata.impl.dvdproflocal.LocalDVDProfMetaDataProvider,org.jdna.media.metadata.impl.themoviedb.TheMovieDBMetadataProvider,org.jdna.media.metadata.impl.mymovies.MyMoviesMetadataProvider
/metadata/wordsToClean=1080p,720p,480p,1080i,720i,480i,dvd,dvdrip,cam,ts,tc,scr,screener,dvdscr,xvid,divx,avi,vrs,repack,mallat,proper,dmt,dmd,stv,HDTV,x264

Do I need to add something to my properties file to get it to autoupdate?

[stuckless]

@farm - Some people confuse the DVD profile plugin in SageMC with the DVD Profiler plugin for bmt. They are totally different solutions. So, any reference to DVD Profiler in SageMC has nothing to do with bmt and vice versa.

Your properties look fine. Although you can get rid of the compositeMetadataProvider stuff, if you want. The current bmt provides a composite provider for dvdprofiler, called dvdprofiler-2, which you are using instead of the one that you configured.

As for the automatic plugin.... It has to be configured on the sagetv server and not on the client. You'll need to check your Sage.properties on the server and ensure that that the plugin is installed as per the manual instructions.

Once the plugin is installed (and the server restarted), you can test the plugin by adding a new media item to your library, and forcing a sagetv refresh. If metadata is not showing up, then look in the SageTV log file (sagetv_0.txt) and look for lines that have BMT in them. When a media file is scanned, it will be logged there.

[matterw]

Stuckless: Thanks for the quick fix (i.e., replacing sagex apis). That did it. It hangs at the end of the run, but it works for the time being. I'll keep my eye out for a permanent fix.

[Farm]

@stuckless
Thanks. It seems to be working now on my Sage Client, but not on my HD200. The Central Fanart folder is that same and BMT is installed on the server, but no fanart shows for any video on the HD200. I'll have to troubleshoot that.

However, on the Client (where Fanart currently works) if the videos are sorted by "All Videos" then I see the cover art and a background fanart as well. If I sort the videos by "Genre", the background fanart will show up as I scroll, but I have NO cover art for the videos. ????? Any ideas on that?

I'm having a whole separate issue now of the Client not being able to access the shared folders for playback and that has turned into a troubleshooting nightmare. I've tried every iteration of user name, shared permission, etc. and no dice.

One of these days it will all work.

[stuckless]

Originally Posted by Farm

@stuckless
However, on the Client (where Fanart currently works) if the videos are sorted by "All Videos" then I see the cover art and a background fanart as well. If I sort the videos by "Genre", the background fanart will show up as I scroll, but I have NO cover art for the videos. ????? Any ideas on that?

Unfortunately I can't offer any advice on that. Maybe you could post something the SageMC forum about that.

[rmccoy]

Hey Stuckless, any idea on an ETA for the 3.0 beta? I'm about to do an initial setup on my media server for the first time, and I think it would be a heck of a lot easier with that web based UI configuration... If it's going to be a while still, let me know, and I'll just setup using the current iteration. Thanks!

[stuckless]

I'm going to try and get a beta out sometime this weekend... I have a beta now that includes the configuration, but I'm currently adding in Scan support and metadata editing. Maybe i'll release a beta build with just the configuration section, and then release the scan/editing stuff as another beta build later.

[jaminben]

Cool, can't wait to try the configuration stuff.

[rmccoy]

Dang man, this sounds great! Can't wait to try it out! We all really appreciate all the work you've put into developing this great scraper.