Utility: Batch Metadata Importer 3.x

[stuckless]

Originally Posted by tmiranda

Sean, Did you confirm that that plugin is called when new TV shows are recorded? Your statement above seem to indicate that you are referring to imported video (which has always worked fine for me) but not new recorded TV.

Thanks,

Tom

Yes, on my setup, the automatic plugin is being called on recorded shows.

[redman]

I am trying to narrow down what could be causing me the issues where the java fails to run from sage (works from command line)[running 2 or 3]

My log file suggest that there is a java conflict.

Does anyone have bmt running automatically who also has XMLTV installed (I am using the Demedius plugin) which also need xerces.jar ??

Error from bmt:
2009-06-18 12:00:15,285 ERROR [Seeker] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:161) - Failed while performing Automatic Metadata/Fanart lookup
java.lang.AbstractMethodError: org.apache.xerces.dom.DeferredElementImpl.getTextContent()Ljava/lang/String;
at org.jdna.media.metadata.impl.xbmc.XbmcScraperParser.processRegexps(XbmcScraperParser.java:81)
at org.jdna.media.metadata.impl.xbmc.XbmcScraperParser.processRegexps(XbmcScraperParser.java:76)
at org.jdna.media.metadata.impl.xbmc.XbmcScraperParser.processRegexps(XbmcScraperParser.java:76)
at org.jdna.media.metadata.impl.xbmc.XbmcScraperParser.parseScraper(XbmcScraperParser.java:48)
at org.jdna.media.metadata.MediaMetadataFactory.<init>(MediaMetadataFactory.java:84)
at org.jdna.media.metadata.MediaMetadataFactory.getInstance(MediaMetadataFactory.java:33)
at org.jdna.media.util.AutomaticUpdateMetadataVisitor.<init>(AutomaticUpdateMetadataVisitor.java:33)
at org.jdna.sage.MetadataUpdaterPlugin.extractMetadata(MetadataUpdaterPlugin.java:98)
at sage.media.format.g.a(Unknown Source)
at sage.media.format.g.a(Unknown Source)
at sage.b1.nv(Unknown Source)
at sage.b1.n0(Unknown Source)
at sage.ce.kT(Unknown Source)
at sage.ce.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2009-06-18 12:00:15,285 DEBUG [Seeker] org.jdna.sage.MetadataUpdaterPlugin (MetadataUpdaterPlugin.java:167) - Returning NULL Properties for File: E:\TheAmazingRaceAsia-S3Ep6-9793514-0.ts

[stuckless]

redman, i added a bug for this this. Not sure there is anything that I can do short of moving away from jaxp and using dom4j or something else, which I may do anyways because its more efficient at parsing large xml files like dvd profiler and mymovies. But this won't happen anytime soon. But the error is logged, so that I don't totally forget about it.

[loomdog32]

I only posted the first url to show the 403 error, as I didnt know if it was a problem with Jetty or something else..

Of the urls you posted, the https doesnt do anthing for me. the other gives a 404 error (I can get a screen cap if you want, but its just a generic Jetty 404 error page)


--edit--

I figured it out.. User Error.. I copied the files wrong..

[tmiranda]

Originally Posted by stuckless

Yes, on my setup, the automatic plugin is being called on recorded shows.

OK, sanity check. Here is what I did on my system:

- Shutdown the SageTV service
- extracted the ZIP archive to the /SageTV/SageTV directory
- double checked that the two different lines necessary are present in the Sage.Properties file
- imported the STVi for the default STV (on an extender)
- on the Metadata / Fanart screen:
- set fanart enabled to Yes
- configured the central fanart directory
- enabled the automatic plugin
- did not do anything on the Metadata / Fanart sources tab

I'm running the latest .17 beta and version 2.3.1 of your plugin.

I'm using the default STV with the Malore menus enabled.

My Sage system is probably not corrupt because I just rebuilt the server a few weeks ago.

I'm running WindowsXP Home SP3 with all of the latest updates.

I'm running the next to latest java. (I see there is a new version available but I have not tried it yet.)

Do you see anything I've messed up? How can I debug this further?

If all else fails, can you recommend a good Exorcist?

Tom

[sflamm]

tmiranda,

Sean has promised to send the metadata.properties and sage.properties files he used to get my image running successful.

Hopefully these will shed some light on your configuration issue.

[redman]

Originally Posted by stuckless

redman, i added a bug for this this. Not sure there is anything that I can do short of moving away from jaxp and using dom4j or something else, which I may do anyways because its more efficient at parsing large xml files like dvd profiler and mymovies. But this won't happen anytime soon. But the error is logged, so that I don't totally forget about it.

Hi Stuckless,
FYI: I have done a fresh install of sage, and used a different XMLTV plugin that doesn't require xerces.jar, and BMT is now automatically adding in metadata from sage now.

**COOL**

So, looks like it was some conflict with xerces. Thanks for your help.

[mkanet]

Currently, all the fanart is still being automatically downloaded to the same directory as where each respective mpg file is located. For example, fanart jpgs are being saved to the TV Recordings folder; and, also in each respective Video imports folder without creating the TV and movies folders (all files mixed together) in each separate folder that has mpg's (including TV recordings).

Webui fanart settings look correct as far as I can tell. If I manually do an on Demand save of the fanart/metadata, then the fanart will be copied to E:\fanart (which works correctly; and, always has).

The above-mentioned behavior has happened in both version 2.3.1 (although it happened only once) and consistently in version 3.0.

It looks like for whatever reason, the automatic feature can't read the fanart location setting defined in the properties file (which the same setting works correctly for manual On Demand saving.

Properties:
bmt/plugin/enabled=true
phoenix/mediametadata/fanartCentralFolder=E\:\\Fanart
phoenix/mediametadata/fanartEnabled=true

Originally Posted by stuckless

3.0 uses a different configuration setup than the 2.x, which is why I also include the web ui for configuring it as well. If you have 3.0 plugin installed, then use the web ui, and click on Configure / Metadata Tools / Application Configuration and set your fanart folder there. Based on your description, it sounds as if the fanart folder is not set.

As for the plugin/on-deman, they are different entry points in to the same system. They both use the same configuration, etc, but "how" they invoked and the data they are passed is what makes them different. Once a search happens in either, they are sharing the same code, etc.

You may be seeing a difference in the results, if, you are running a client connected to the server, and you've set the fanart folder on the client ui, but it's not set on the server. In that case, the on-demand will work fine since the client will have it's fanart folder set, but the automatic plugin, which is invoked on the server, will not have its fanart folder set, and will download fanart to the local directory.

The Sage.properties property for the central fanart folder is phoenix/mediametadata/fanartCentralFolder

EDIT:
The 3.x release no longer uses the metadata.properties file, it stores it configuration in the Sage.properties file.

[loomdog32]

Now that I got the webUI running, I cant get my desired goal.. To import videos into Recorded TV.. Is this possible with BMT?? If so, could someone please give me a step by step how to on how to acomplish this...


Thanx


I dont think its pulling metadata for TV either.. My files are done as

Show Name - SXEXX - Eposide Title
Show Name - EE - Eposide Title

I might understand more with a show like Transformers Beast Wars Neo, but Im having the same problems with shows like House and CSI

[ShortySeanbo]

This looks like a great utility! I think I'm very close to using to its full potential.

After much struggling I upgraded from SageMC 6.3.8 to 6.3.9. I then enabled fanart and left the directory as the default [SageDir]\STVs\Phoenix\Fanart. This seemed to work. I created a TV folder and a Mythbusters\backgrounds\ folder and put an image in there and it was displayed as the background.

I then extracted and imported the Metadata Importer and loaded into SageMC. I ran the file at the command line and was greeted with a bunch of errors about how it couldn't find a server (I only have one machine running Sage, it is the server). I tried to install the remote APIs, but they didn't seem to work. I finally found a patched version in this thread that gives up looking for the server and that worked!

So the Metadata Importer runs through a huge directory of recorded files, updates metadata (though what it was updating, I do not know), and downloaded a bunch of fanart jpgs.... to the folder where all the files were.

Perplexed, I ran the --showProperties flag and low and behold the fanart directory seemed to be blank (even though it showed up just fine in SageMC). I tried to specify the directory manually using setProperty, but that didn't seem to work either.

First question:
Is there a way to automagically get the files in the right place, and more importantly, in the directory structure they need to be?

Second question: The utility seemed to download the identical image for every episode of a show. As they are jpgs, I'm not overly worried about storage usage, but I wanted to make sure this was the desired behavior, as opposed to downloading one image per show (rather than per episode, if it is the same image).

Thanks,
-Sean

[mkanet]

Sean (ShortySeanbo), in case you haven't noticed you have the same exact problem I do.

[stuckless]

Originally Posted by loomdog32

Now that I got the webUI running, I cant get my desired goal.. To import videos into Recorded TV.. Is this possible with BMT?? If so, could someone please give me a step by step how to on how to acomplish this...

Currently, bmt has a beta option for allowing this to happen for on demand searches only. To enable this option, go into the configuration ui, search for "tv" and you should see an option, Import TV as Sage Recordings. If you enable this, then if you do on-demand searches for you tv shows, then they'll be imported as Sage Recordings. This feature is experimental. I have never personally used it because I want my other tv shows to be imported as recordings. Eventually in bmt 3.x I'll enable this as a feature for the automatic plugin as well. But I'd like to get more feedback on whether or not it's working as expected before I enable that. I'd also like to be able to generate "undo" files, in case you imported something that accidentally, etc. None of those things exist yet.

[stuckless]

Originally Posted by ShortySeanbo

So the Metadata Importer runs through a huge directory of recorded files, updates metadata (though what it was updating, I do not know), and downloaded a bunch of fanart jpgs.... to the folder where all the files were.

Perplexed, I ran the --showProperties flag and low and behold the fanart directory seemed to be blank (even though it showed up just fine in SageMC). I tried to specify the directory manually using setProperty, but that didn't seem to work either.

As noted in other posts.... bmt 2.x uses a separate configuration file and not the Sage.properties. When you configure bmt from withing the STV it gets written to the Sage.properties and not the metadata.properties. So because of this, you need manually specific the fanart dir on the commandline (--fanartFolder=) or put it in the metadata.properties. (the 3.x beta just uses Sage.properties, and can be configured via the web ui)

Is there a way to automagically get the files in the right place, and more importantly, in the directory structure they need to be?

when running from the command line use --fanartFolder=YOUR_FOLDER and it will go to the correct place.

The utility seemed to download the identical image for every episode of a show. As they are jpgs, I'm not overly worried about storage usage, but I wanted to make sure this was the desired behavior, as opposed to downloading one image per show (rather than per episode, if it is the same image).

If your shows are recognizes as tv shows, such as, "SeriesName S##E## Episode Title.avi", then fanart will downloaded per season and per show. There may be duplicates and the tool does not try to find out if season 1 fanart is the sames as season 2, etc. So, if i understand the question correctly, then I think this is normal.

[PiX64]

Sorry if this has already been asked and answered, but i am having a issue getting the bmt webpage loaded when the sageclient is not up and running on the server. My server is headless, and i don't use it as a client anymore. If i start the service and let it run, everything works fine as far as accessing other websites (neilms plugin) etc. I cannot however access the /bmt page UNLESS i remote in and start the client GUI on the server.

any thoughts?

[stuckless]

I'm not a windows person... but are you running vista? did you install the bmt code as the same user as the sage service? Are you sure you've installed the bmt code in the correct sagetv directory? ie, is the sagetv client and server different directories?

[QueOnda]

BMT 3.0 --

Phoenix
Version

1.31
sagex.api Version

6.5.17-2
Fanart Enabled

true
Fanart Folder

C:\Program Files\SageTV\SageTV\STVs\Phoenix\Fanart

Metadata Tools
Version

3.0-beta-3
Automatic Plugin Enabled (Server)

true
Custom Fields Configured

true
Last Scan Date

Tue Jun 23 04:51:02 PDT 2009
Total Scanned

4
Last Scanned MediaFile

SesameStreet 4426325 0

SageTV
Version

6.5.17.213
Required Version

6.5.16
Java Version

1.6.0_13
Server

true
Recordings Used Diskspace

764.73G
Library Import Scan in Progress
false

When I go to the "C:\Program Files\SageTV\SageTV\STVs\Phoenix\Fanart" There is a TV folder with some shows. (good)

Then I go into one of the folders "C:\Program Files\SageTV\SageTV\STVs\Phoenix\Fanart\TV\Kendra\Banners" There is a 98371-g.jpg (0K file) and an images (name only no extention) 1k size.

Seems making files but no data. Can someone point me somewhere to start to debug?

[PiX64]

That did the trick. I changed the user used by the service, and all is well. Thanks for your help stuckless.

I would also like to say that i have been waiting to try these set of tools for way to long. This is by far one of the best plugins and additions for sagetv out there. Your right up there with Neilms web interface.

Great job! Please let me know if i can help you out in any way, coding, testing, documentation, whatever.

~Mike

[stuckless]

Originally Posted by QueOnda

BMT 3.0 --

When I go to the "C:\Program Files\SageTV\SageTV\STVs\Phoenix\Fanart" There is a TV folder with some shows. (good)

Then I go into one of the folders "C:\Program Files\SageTV\SageTV\STVs\Phoenix\Fanart\TV\Kendra\Banners" There is a 98371-g.jpg (0K file) and an images (name only no extention) 1k size.

Seems making files but no data. Can someone point me somewhere to start to debug?

Seems that others have pointed to this issue as well. There is currently a bug open for 0 byte files. Hopefully not ALL files are zero byte?

[stuckless]

Originally Posted by PiX64

That did the trick. I changed the user used by the service, and all is well. Thanks for your help stuckless.

I would also like to say that i have been waiting to try these set of tools for way to long. This is by far one of the best plugins and additions for sagetv out there. Your right up there with Neilms web interface.

Great job! Please let me know if i can help you out in any way, coding, testing, documentation, whatever.

~Mike

Thanks for the great comments. You are helping by using and testing this software But, the source is available, and feel free browse the source, suggest enhancements, update the wiki, find bugs, fix bugs, etc. You can even submit patches if you like. I can't promise that I'll accept them, but haven't rejected anyone so far

[QueOnda]

Originally Posted by stuckless

Seems that others have pointed to this issue as well. There is currently a bug open for 0 byte files. Hopefully not ALL files are zero byte?

Cool. I do have about 10 folders and the ones I looked at (most of them) all have zero byte.

Stuckless, where can I find documentation on getting this setup correctly? I have found this: http://docs.google.com/Doc?id=dff28jvm_13f4rf57f6 Still reading and re-reading to come up to speed.

I trying to setup BMT by pointing to a exported .xml (my moves file) but don't know if it actually doing anything. Need to play around some more, I know I don't have it setup correctly.

Anyhow, MyMovies 2.55 has "option to store meta-data for SageTV" Anyone know what that's about?