Utility: Batch Metadata Importer 3.x

[stuckless]

if you just extract the beta3 into the sage root, then you don't need to re-import the stv plugin at all. The only time you need to re-import he stv plugin is when you change/update the stv.

[gplasky]

Great job on this. What I'm seeing in my status is a failing for all of my DVDs. (About 1104 failures)The first line when I hover over it says "Scan did not produce a property file." and the second line says "Nothing found for Query: SearchQuery; Type: MOVIE; TITLE: VIDEO;"
Am I doing something wrong? I have DVD Profiler, theMovieDB, IMDB the TVDB as providers. (And of course the rest of the scan errors are TV movies I have recorded.)

Gerry

[matt91]

Originally Posted by babgvant

change <Set name="extractWAR">false</Set> to <Set name="extractWAR">true</Set>
in "Program Files\SageTV\SageTV\jetty\contexts\bmt.xml"

Ah. Worked for me too. I had even tried to extract the war file myself (which I did), but it didn't help.

thanks for the hint.
Matt

[stuckless]

Originally Posted by gplasky

Great job on this. What I'm seeing in my status is a failing for all of my DVDs. (About 1104 failures)The first line when I hover over it says "Scan did not produce a property file." and the second line says "Nothing found for Query: SearchQuery; Type: MOVIE; TITLE: VIDEO;"
Am I doing something wrong? I have DVD Profiler, theMovieDB, IMDB the TVDB as providers. (And of course the rest of the scan errors are TV movies I have recorded.)

Gerry

Sounds like you are doing things right... but "SearchQuery; Type: MOVIE; TITLE: VIDEO" means that bmt thinks the video title is "VIDEO". What's your folder layout look like?

[gplasky]

Originally Posted by stuckless

Sounds like you are doing things right... but "SearchQuery; Type: MOVIE; TITLE: VIDEO" means that bmt thinks the video title is "VIDEO". What's your folder layout look like?

Sage Import directory is \\MAIN\Videos\MainMovie
and also \\MAIN\Videos\FullDisc

Folders in MainMovie and FullDisc are DVD titles.

\\MAIN\Videos\MainMovie\300
\\MAIN\Videos\MainMovie310 to Yuma
\\MAIN\Videos\MainMovie\A Beautiful Mind

\\MAIN\Videos\FullDisc\Dark CIty
\\MAIN\Videos\FullDisc\Dead Space Downfall
\\MAIN\Videos\FullDisc\The Incredible Hulk


From the mailcheck log:

2009-06-13 16:55:45,345 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\FullDisc\The X-Files-I Want To Believe\VIDEO_TS
2009-06-13 16:55:45,455 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\FullDisc\Total Transformation\Jump Start DVD\VIDEO_TS
2009-06-13 16:55:45,549 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\FullDisc\Total Transformation\Workshop DVD\VIDEO_TS
2009-06-13 16:55:45,658 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\FullDisc\Winsor McCay-The Master Edition\VIDEO_TS
2009-06-13 16:55:45,830 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\FullDisc\Wonder Woman\VIDEO_TS
2009-06-13 16:55:45,939 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\FullDisc\You Don't Mess with the Zohan\VIDEO_TS
2009-06-13 16:55:46,080 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\MainMovie\10th & Wolf\VIDEO_TS
2009-06-13 16:55:46,174 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\MainMovie\11 14\VIDEO_TS
2009-06-13 16:55:46,377 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\MainMovie\16 Blocks\VIDEO_TS
2009-06-13 16:55:46,455 ERROR [LibraryImporter] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:152) - Failed to Fetch Metadata for Media: \\MAIN\Videos\MainMovie\21\VIDEO_TS

And so on.

[mkanet]

Forgive the newbie question. But after I import BMI and after select the folder and settings it to automatic, I am not sure how to get it to automatically load fanart and update metadata for all the imports video I have; even after I select to scan import library. Am I missing something? What sageTV imports can I try to see fanart in action?

Is ther a missing component to populate all metadata for all my imported videos? So far, I have to manualy select "edit video info" then select Metadata/Fanart, then select the one it finds, then finally I get the fanart. It's not feasible to do this for every single movie.

I even put this in my properties page: force_full_content_reindex=true

What is causing these video files not to automatically get their metadata from BMI? How long does it take to update a few dozen videos? Mine doesnt appear to be doing anything automatically.

Thanks in advance!

This is what I did so far:
1) copy the BMI install files to the respective folders,
2) In SageMC import the MetadataFanartTools-SageMC.xml, SageMC/Extras/Batch Metadata Importer=Automatic
3) SageMC/Main/Global UI Options/Display/FanArt Options/Fan Art Central Path=E:\Fanart

The only way to get fanart is to manually go into imported videos, Edit Video Info, select Metadata/Fanart, choose the video name. Thats the only way my fanart central folder receives the files. I dont know how to make this automatic for any of the imports that support fanart. I would like all my metadata to be populated automaticaly for all recognized videos/tv shows. So far anything I do requires a manual "Edit video Info" under video imports; which I know isnt the right way to do it.

[stuckless]

Originally Posted by gplasky

Sage Import directory is \\MAIN\Videos\MainMovie
and also \\MAIN\Videos\FullDisc

Folders in MainMovie and FullDisc are DVD titles.

\\MAIN\Videos\MainMovie\300
....

The good news is that you are still doing everything right... unfortunately... I'm not There is bug in the dvd processing code.

[rmccoy]

Well this is a bit frustrating... the web configuration page was loading just great for a while earlier today! But when I went back to it, now it just shows the directory structure...

Directory: /bmt/
BMTWeb.css 1365 bytes Jun 11, 2009 9:35:12 PM
BMTWeb.html 1868 bytes May 25, 2009 8:51:22 PM
WEB-INF/ 0 bytes Jun 13, 2009 10:11:51 AM
bmtweb/ 0 bytes Jun 13, 2009 10:11:50 AM

I've restarted my computer, and sagetv, even reinstalled BMI... any idea what would cause this?

[mkanet]

Can anyone who uses fanart please help? I was hoping to get it working tonight. Maybe I also need mediascraper to make BMI automatic?

Originally Posted by mkanet

Forgive the newbie question. But after I import BMI and after select the folder and settings it to automatic, I am not sure how to get it to automatically load fanart and update metadata for all the imports video I have; even after I select to scan import library. Am I missing something? What sageTV imports can I try to see fanart in action?

Is ther a missing component to populate all metadata for all my imported videos? So far, I have to manualy select "edit video info" then select Metadata/Fanart, then select the one it finds, then finally I get the fanart. It's not feasible to do this for every single movie.

I even put this in my properties page: force_full_content_reindex=true

What is causing these video files not to automatically get their metadata from BMI? How long does it take to update a few dozen videos? Mine doesnt appear to be doing anything automatically.

Thanks in advance!

This is what I did so far:
1) copy the BMI install files to the respective folders,
2) In SageMC import the MetadataFanartTools-SageMC.xml, SageMC/Extras/Batch Metadata Importer=Automatic
3) SageMC/Main/Global UI Options/Display/FanArt Options/Fan Art Central Path=E:\Fanart

The only way to get fanart is to manually go into imported videos, Edit Video Info, select Metadata/Fanart, choose the video name. Thats the only way my fanart central folder receives the files. I dont know how to make this automatic for any of the imports that support fanart. I would like all my metadata to be populated automaticaly for all recognized videos/tv shows. So far anything I do requires a manual "Edit video Info" under video imports; which I know isnt the right way to do it.

[gplasky]

Originally Posted by stuckless

The good news is that you are still doing everything right... unfortunately... I'm not There is bug in the dvd processing code.

Thanks for the info. And good luck on the fix. I know how busy you must be.

Gerry

[stuckless]

@mkanet - Is the bmt code installed on the sagetv server? Have you verfied the sage.properties on the server to see if the metadata plugin class is installed? If you run the bmt 3.x beta, then the web status screen will tell you if it's installed or not. Also the web status screen will show you last 5 errors that the automatic encountered. If the automatic plugin is showing a last scan date of many years ago, then the plugin is not being invoked on new media.

[GollyJer]

Originally Posted by rmccoy

Have you considered my proposition of including the option to auto import TV shows into the "Recorded TV Shows" menu? I know it's difficult to undo that action, but I haven't had a problem with it as of yet!

Originally Posted by stuckless

I've considered it... it's not implemented, yet.

This would be really awesome. I prefer BMT but have to use mediaScraper for the AddToSageDB functionality.

[redman]

Hi,
I am able to get the 3.0 BMT web page up, but cant get any updates to my sage data. (could only manually scan 2.0 bmt as well). On the status screemn Last scan is showing 1970, and no errors are showing up on this screen.

I have enable full debugging.
I noticed this error in the logs:

2009-06-15 00:18:04,733 ERROR [btpool0-7] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files\SageTV\SageTV\STVs\Phoenix\Configuration\Phoenix.xml

I can't seem to find this file anywhere. is it needed?

As something else of note, I am also using XMLTV, which uses some old Xerces.jar any conflicts with this at all perhaps?

Thanks for your awesome work.

[stuckless]

Originally Posted by redman

Hi,
I am able to get the 3.0 BMT web page up, but cant get any updates to my sage data. (could only manually scan 2.0 bmt as well). On the status screemn Last scan is showing 1970, and no errors are showing up on this screen.

That's an indication that the automatic plugin is NOT being hit. Does it show as being installed? (Automatic Plugin Enabled (Server))

I have enable full debugging.
I noticed this error in the logs:

2009-06-15 00:18:04,733 ERROR [btpool0-7] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files\SageTV\SageTV\STVs\Phoenix\Configuration\Phoenix.xml

I can't seem to find this file anywhere. is it needed?

Not needed. BMT uses the phoenix configuration metadata apis, and as such, if it's there, it will be loaded, if not, then shouldn't matter. I assume you are able to "configure" the other things using the web ui, such as log4j, Metadata Tools, etc.

As something else of note, I am also using XMLTV, which uses some old Xerces.jar any conflicts with this at all perhaps?

If you are not seeing any configuration in the configuration screen, then perhaps this could be the issue.

[mkanet]

Stuckless, or anyone could you please help me? I would love to be able to automatically download fanart without manually having to select, "Metadata Fanart" and manually selecting the online name of the move/show. I am so close....

[robogeek]

Originally Posted by mkanet

Stuckless, or anyone could you please help me? I would love to be able to automatically download fanart without manually having to select, "Metadata Fanart" and manually selecting the online name of the move/show. I am so close....

Are you using the 3.0 beta? Are your imported videos ripped DVD in VIDEO_TS folder structure? If so, that's the problem...there's a bug in 3.0 beta.

If you're using the 3.0 beta, check the BMT jetty web page to see if any errors or problems are reported on the Status page. If you're using the 3.0 beta or a previous version, also check the BMT log file (mailcheck.log) in the SageTV directory for errors.

BMT v2.3.1 was working great for me. If you're trying out the beta, maybe you should try v2.3.1 if you want something more stable.

[mkanet]

I'm using 2.3.1 I just followed the directions to install with only bare minimum settings (as described in all my posts requesting for help). TV recordings, Imports, everything will only have fanart if I manually select to do an "on demand" download of metadata.

Originally Posted by robogeek

Are you using the 3.0 beta? Are your imported videos ripped DVD in VIDEO_TS folder structure? If so, that's the problem...there's a bug in 3.0 beta.

If you're using the 3.0 beta, check the BMT jetty web page to see if any errors or problems are reported on the Status page. If you're using the 3.0 beta or a previous version, also check the BMT log file (mailcheck.log) in the SageTV directory for errors.

BMT v2.3.1 was working great for me. If you're trying out the beta, maybe you should try v2.3.1 if you want something more stable.

[stuckless]

Originally Posted by mkanet

I'm using 2.3.1 I just followed the directions to install with only bare minimum settings (as described in all my posts requesting for help). TV recordings, Imports, everything will only have fanart if I manually select to do an "on demand" download of metadata.

As per my previous post to you.... Is bmt running on the sagetv server? Is the server configured correctly as per the manual installatation instructions in the installation wiki. (ie, verify in the server's sage.properties includes then mediafile_metadata_parser_plugins property set correctly)

If the answers as "yes" and "yes", then the next step, as robogeek pointed out, is to look in the mailcheck.log for issues. You may have turn on debug logging. My first post contains a link to how configure logging in 2.3.

Also, you need to tell sage to rescan its media library when you add new media, or wait until sage does it on its interval.

[mkanet]

Oops.. sorry Stuckless, I didnt see this reply until now. I have the bmt code installed on the server. I am trying to get it work on the SageTV client thats on the same machine as the server as well. Can you check to see my properties files are okay? I am using the 2.xx version of BMT. I know that metadata download of fanart does work since I can do it manually (and the files get downloaded into the central fanart dir.

Edit: I can submit logs if everything looks like as far as properties go...

Thanks again so much for helping..

Originally Posted by stuckless

@mkanet - Is the bmt code installed on the sagetv server? Have you verfied the sage.properties on the server to see if the metadata plugin class is installed? If you run the bmt 3.x beta, then the web status screen will tell you if it's installed or not. Also the web status screen will show you last 5 errors that the automatic encountered. If the automatic plugin is showing a last scan date of many years ago, then the plugin is not being invoked on new media.

[stuckless]

the properties look ok... the server (sage.properties) does have the automatic plugin installed. So the test would be to add a new video file to the library, tell sage to "refresh" its library, and then check the logs (maillog.log, and maybe sagetv_0.txt)