Utility: Batch Metadata Importer 3.x

[PLUCKYHD]

Originally Posted by stuckless

I'll to check the "ignore folders" option to see if it's broken. The idea, is that they should not be included in your scan, but it they are, then it's broken



That is correct. It's the "display title" for the item. The mask is used when calculating the display title, and it gets written to the sagedb or the properties file.

That is what I thought just making sure thanks for the clearification

[Rob]

Suckless,

Thanks so much for your work on Metadata. I installed sagemc, your tools and Plucky's movie wall a couple of weeks ago and it has been a great enhancement you my families sagetv experience. Your web interface is great! I just posted in Plucky's tread to thank him too. Not sure where to post to thank MeinMaui, so hoping he sees these.

[stuckless]

Originally Posted by Rob

Suckless,

Thanks so much for your work on Metadata. I installed sagemc, your tools and Plucky's movie wall a couple of weeks ago and it has been a great enhancement you my families sagetv experience. Your web interface is great! I just posted in Plucky's tread to thank him too. Not sure where to post to thank MeinMaui, so hoping he sees these.

Thanks Rob! Glad it's working out for you.

[evilpenguin]

Hey Sean,

I just got around to loading up the a 3.0 beta for the first time and i'm having some trouble. First, here's my setup

Phoenix Version: 1.38
sagex.api Version: 6.5.19
Fanart Enabled: true
Fanart Folder: D:\Fanart
	
Metadata Tools
Version: 3.0-beta-9
Automatic Plugin Enabled (Server): false
Custom Fields Configured: true
Last Scan Date: Wed Dec 31 16:00:00 PST 1969
Total Scanned: 0
	
SageTV
Version: 6.6.2.218
Required Version: 6.5.16
Java Version: 1.6.0_04
Server: true

Jetty: v1.6

This is a fresh install of both Jetty and BMT and I followed Sflamm's instructions.

1) Shutdown SageTV and unzip Jetty v1.6, and configure .properties file
2) Launch SageTV, verify Jetty is working by going to localhost:8080
3) Shutdown SageTV and unzip BMT v3.0 beta 9 and configure .properties file
4) Launch SageTV
5) Go to localhost:8080/bmt and I get the "Failed to get status: SageTV/Metadata Tools/Phoenix" errors.

However if I...

1) Shutdown SageTV and unzip Jetty v1.6, and configure .properties file
2) Launch SageTV, verify Jetty is working by going to localhost:8080
3) Shutdown SageTV and unzip BMT v3.0 beta 9 and DON'T set mediafile_metadata_parser_plugins
4) Launch SageTV
5) Go to localhost:8080/bmt/ and I see the status page load properly
6) Shutdown SageTV and set mediafile_metadata_parser_plugins=org.jdna.sage.MetadataUpdaterPlugin
7) Launch SageTV, go to localhost:8080/bmt/ and I get the "Failed to get status for: SageTV/Metadata Tools/Phoenix"

Then if I try and remove

mediafile_metadata_parser_plugins=org.jdna.sage.MetadataUpdaterPlugin

and go to localhost:8080/bmt/ I get a blank page. I've attached all the logs I could capture.

Any thoughts?

-EP

[stuckless]

EP,

I wish I had better news... The logs that you posted contained no indication of an error (it would have been nice to see an exception in there for tracking purposes)

I can't even begin to wonder why the addition/removal of the MetadataUpdaterPlugin would have an effect on the web ui, since the 2 are in no way related.

a couple things to try....
1. In your browser, do a force reload of the page. (hit the reload button while holding down the shift key). This will force the browser to load all images, css, and javascript instead of using a cached version.
2. If you are using IE, then try the page using FF. If you use FF, then try the page with IE.
3. delete the extracted BMT directory (webapps/bmt/) and then restart the sage server. It should then extract the bmt webapp again.

[evilpenguin]

Originally Posted by stuckless

I can't even begin to wonder why the addition/removal of the MetadataUpdaterPlugin would have an effect on the web ui, since the 2 are in no way related..

Yeah, I figured that was really weird.

No worries, i'll keep hammering away at it until I can isolate the problem a little more.

[razrsharpe]

Sean,
Id like to make a feature request

Is it possible to have 2 TV Title Masks? One for when you are importing the videos into the Sage Recording db (default = Title) and one for when you are not (Default = Title - S##E## - Episode Title .... what it is currently).
My thinking for this is two fold. First, when importing non-sage recorded TV into the TV db sage groups titles by GetAiringTitle(), so when the title mask is set to more then just "Title" episodes are not grouped properly in My TV. Conversely when not importing into the sage recording db it is beneficial to have the full title mask when browsing in the default video library so you can distinguish between episodes.

The second reason is much more selfish In TVE i want to be able to group correctly no matter where the episode metadata came from. I currently have it working however the only way i can think to do it is call GetMediaFileMetadata(Mediafile, "MediaTitle") on every episode and only use the AiringTitle if MediaTitle does not exist (length = 0). This works however it is MUCH slower compared to checking if the file is a TVFile; if true return GetAiringTitle, if false return the MediaTitle.

Short answer: The benefit of the 2 title masks is we get correct grouping when video is imported into My TV and meaningful titles when browsing non-imported video in the video library.

[stuckless]

Originally Posted by razrsharpe

Sean,
Id like to make a feature request

Is it possible to have 2 TV Title Masks? One for when you are importing the videos into the Sage Recording db (default = Title) and one for when you are not (Default = Title - S##E## - Episode Title ....

Funny you should ask. I haven't acutally touched phoenix or bmt in over a week. I'm gearing up to go on vacation, so I'm trying to get a bunch of stuff done for work, etc.... but, then last thing I was working on, was the "Save" function in the web ui, where it would import "TV" metadata into the sage recordings (if requested), and the thing that I noticed was that in the Sage Recordings, the "Title" should just be the Show name and not include the Season Episode etc.

What I was going to to, was have 2 title masks, as you mentioned, but also have a episode title mask as well, that would default to "${EpisodeTitle}", but you could set it to "S${SeasonNumber} E${EpisdoeNumber} ${EpisodeTitle}" if you wanted the episode title (in the sage recordings) to show the season and episode info as well.

Just to give you some indication of timing... August is write off for me... I have 2 weeks of vacation, no internet (who does that???), so I won't have another bmt release until sometime in september.... but at that time, I'm hoping to have the metadata editor complete.

[razrsharpe]

Originally Posted by stuckless

Funny you should ask. I haven't acutally touched phoenix or bmt in over a week. I'm gearing up to go on vacation, so I'm trying to get a bunch of stuff done for work, etc.... but, then last thing I was working on, was the "Save" function in the web ui, where it would import "TV" metadata into the sage recordings (if requested), and the thing that I noticed was that in the Sage Recordings, the "Title" should just be the Show name and not include the Season Episode etc.

What I was going to to, was have 2 title masks, as you mentioned, but also have a episode title mask as well, that would default to "${EpisodeTitle}", but you could set it to "S${SeasonNumber} E${EpisdoeNumber} ${EpisodeTitle}" if you wanted the episode title (in the sage recordings) to show the season and episode info as well.

excellent, sounds perfect now i dont have to do anything special in TVE to make sure shows group correctly

Just to give you some indication of timing... August is write off for me... I have 2 weeks of vacation, no internet (who does that???), so I won't have another bmt release until sometime in september.... but at that time, I'm hoping to have the metadata editor complete.

sounds good, enjoy your vacation

[kevandcan]

I installed BMI (using the documented install instructions) and everything appeared to work fine. I went into the web admin console and ran a full scan on my media and TV directories.

Once it was complete, I went into my TV recordings, and a bunch of the recordings (Globetrekker) are now missing. It only shows 1 of the 12 I have.

Additionally, the Fanart only shows for single word shows (e.g. Lost, Dora, etc). I assume this has to do with the naming convention of my files (FileName-date.mpg per the default Sage install. Any way to change this so it automatically names files so BMI picks them up)?

Finally, when I try to search IMDB it always fails now saying it can't find any results (e.g. Search "The Office" and it got zero results, same with everything else). Not sure what is happening, but would love to get this working.

Appreciate any help you can provide.

EDIT: upon reading all 67 pages of this thread, I figured out the manual STVi install (wasn't noted in install instructions linked to on first page) and that seems to work for the TV recordings. Now I need to get Movie recordings done by sageTV under Recorded TV to work. (see post below)

[VikingCrown]

Quick Question, If I want to bring in my TV DVD's. do I have to enter them in DVD Profiler, or will 3.0 pull the info from the moviedb ot the tvdb? Also what is the best folder format/naming convention that is reccomended?

[dbone1026]

Originally Posted by VikingCrown

Quick Question, If I want to bring in my TV DVD's. do I have to enter them in DVD Profiler, or will 3.0 pull the info from the moviedb ot the tvdb? Also what is the best folder format/naming convention that is reccomended?

Are you ripping out individual episodes or bringing in the full dvd as is?

[VikingCrown]

Full DVD as is, I only have a few series and I don't really feel like ripping all the episodes off them.

[dbone1026]

Originally Posted by VikingCrown

Full DVD as is, I only have a few series and I don't really feel like ripping all the episodes off them.

I had tried this at first before I decided to start ripping each episode and I think the fanart/coverart just gets pulled in for the main dvd (i.e you will not have separate fanart for each disc within a season, just for the season itself). You would then just drill down to the disc to play. As I mentioned though, I did this a while back so things might have changed.

As a side note, I actually just finished writing a post on my blog about ripping/encoding tv episodes if it is of interest (http://dbone1026.blogspot.com/2009/0...eries-dvd.html)

[kevandcan]

Great news (well, for me at least)! I have BMI almost completely working. It gets the fanart now for TV shows I record, but when I record a movie using SageTV (and it shows up under Recorded TV), it won't grab the metadata or fanart. Not sure why. Is it because SageTV records movies like TV shows and the filename format isn't compatible with BMI?

Any ideas on how to get my recorded movies in SageTV to grab fanart under Recorded TV? (manual doesn't work either)

Also, can't get command line to run (see below)

INFO:
------------

What bmt version 2.x (stable) or 3.x (beta)? 3.9
What SageTV version? 6.6
SageMC or default STV? SageMC
SageMC version? latest
Java version? 1.6
Phoenix version? 1.38

SageTV setup:
Is central FanArt folder enabled? yes
Is central FanArt folder local or a network share? local
Do you have SageTV set up to do automatic scans? no
If yes, how often?
Do the scans import new videos?

BMT setup:
Command Line Tool: I have it
Have you successfully used the command line tool? no
Here's what it says
Exception in thread "main" java.lang.NoClassDefFoundError: org/jdna/metadataupda
ter/MetadataUpdater
Caused by: java.lang.ClassNotFoundException: org.jdna.metadataupdater.MetadataUp
dater
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
Could not find the main class: org.jdna.metadataupdater.MetadataUpdater. Program
will exit.

On-demand Tool:
Have you imported the on-demand plugin (stvi of xml)? yes
Do you have the on-demand plug-in enabled? yes

Automatic tool:
Do you have these 2 properties set in sage.properties? yes

mediafile_metadata_parser_plugins=org.jdna.sage.MetadataUpdaterPlugin 
 custom_metadata_properties=UserRating;EpisodeNumber;EpisodeTitle;MediaTitle;MediaType;OriginalAirDate;SeasonNumber;MediaProviderDataID;DiscNumber

Please give a description of the steps to reproduce your issue:
Can't run command line nor does BMI pull metadata or fanart for movies under Recorded TV (movies recorded using SageTV on TNT, for example ("Harry Potter"). Suspect it might be filename issue?

[stuckless]

@kevandcan - Support for Movies that are recorded has not been added... but it will be. As for the commandline error, when you extracted the bmt tool, did it create a JARs folder and put a bunch of .jar files in there? Also, when you are running the tool, are you running it from a command shell, or by double clicking on the .jar. I'm not sure the doubleclicking will work. Also, I think that razrsharpe posted a .bat that could be tweaked, which might help as well.

@VikingCrown - If your TV DVDs are in the following format then it will be processed as a TV Disc.

YourMediaDir/Dexter S01D01/VIDEO_TS/

You can also have something like...

YourMediaDir/Dexter/Dexter S01D01/VIDEO_TS/
YourMediaDir/Dexter/Dexter S01D02/VIDEO_TS/

The bottom line is that for a TV DVD the folder that contains the the VIDEO_TS or the .vob files will need to contain the Series Name (ie, Dexter), the Season and the Disc #.

The BMT application cannot actually get episode information but it will attempt to season specific fanart.

[razrsharpe]

Originally Posted by stuckless

Also, I think that razrsharpe posted a .bat that could be tweaked, which might help as well.

I've been meaning to repost the bat files and include more of the common operations. See attached.

NOTE: If BMT is not installed at C:\program files\SageTV\SageTV you must modify the BAT files for your path.

[Peter_h]

So I'm just getting started with BMT. Thanks Sean for the great utility.

I have a few questions that I can't seam to find the answers for.

BMT Supports multiple sources to get fan art. Does it use all of them? Meaning, does it search the first source, and if it does not get a hit move on to the second?

How does the DVD profiler tie in? I was previously using DVD profiler and I have not added in the DVD profiler xml location. If it can not find a hit online, does it then go to DVD profiler?

Is there a way to designate metadata sources for specific items? For example, BMT went out and grabbed the metadata for a couple of TV show DVD's. However, the metadata only has episode listings and no episode descriptions. How would I tell BMT/Sage to always use the dvd profiler xml for these dvds?

How can I tell if it is grabbing fanart for my recorded shows? As far as I can tell this does not appear to be working. The web interface is showing the last scheduled scan as when I ran it for the first time. Does this refresh when it scans hourly?

Sorry for all the questions. I read through all of the documentation but it's still not clear for me.

Thanks,
Peter

[kevandcan]

Originally Posted by stuckless

@kevandcan - Support for Movies that are recorded has not been added... but it will be. As for the commandline error, when you extracted the bmt tool, did it create a JARs folder and put a bunch of .jar files in there? Also, when you are running the tool, are you running it from a command shell, or by double clicking on the .jar. I'm not sure the doubleclicking will work. Also, I think that razrsharpe posted a .bat that could be tweaked, which might help as well.

Yes, I extracted all the JARs into the JARs folder (even redid it last night to double check). When I run the commandline, I run it from the "DOS" prompt, not by double clicking. Not sure why it isn't finding the metadata-updater.jar file. It's there and loaded in memory (won't delete when SageTV service is running).

Since the tool works manually in SageTV, I assume (probably incorrectly), that it's calling the metadata-updater.jar from there and it works...

[razrsharpe]

Originally Posted by kevandcan

Yes, I extracted all the JARs into the JARs folder (even redid it last night to double check). When I run the commandline, I run it from the "DOS" prompt, not by double clicking. Not sure why it isn't finding the metadata-updater.jar file. It's there and loaded in memory (won't delete when SageTV service is running).

Since the tool works manually in SageTV, I assume (probably incorrectly), that it's calling the metadata-updater.jar from there and it works...

when running from the dos prompt did you change your working directory to the one with metadatatool.jar in it (ie c:\program files\sagetv\sagetv)? silly question i know but want to be sure.