Utility: Batch Metadata Importer 3.x

[stuckless]

I was hoping it would be easy to add a feature which might tell you which movies you had in a DVD Profiler database which were not represented in the available movies in a directory

I guess following brents logic, you could the use Sage API and get a list of movies, then for each movie, ask for it's File, and if it's File does not exist, then it's one of the missing ones. This all assumes that Sage doesn't "clean" it's database once it detects that a movie is no longer available. This is certainly a feature that I look at adding to my stuff... but I doubt it would be adding in a time that would benefit you currently

[bcjenkins]

I don't see the need to involve SageTV in the search though. It's not like we're querying the IMDB database, it is a small 390ish list of movies. The properties creation tool is very accurate and would be able to determine the 100~ movies I lost on the HD.

I could write a bash script to do this, but I have never been good at regex queries

B

[stuckless]

I could write a bash script to do this, but I have never been good at regex queries

If you send me some reqex expressions, with what you are trying to find, I can try to help you sort out the regex stuff.

I would think that the basic process/pseduo flow for this would be...

for each title in dvd profiler xml ; do
   if [ any properties file contains title match ] ; then
      ignore
   else
      add to missing list
   fi
done

[Brent]

Hey Sean,
Any progress on the UI part of this that you mentioned in your blog a few months ago? Really looking forward to it

[stuckless]

Sorry Brent.... Life, work, and the ocasional sage remote api bug, has been getting the way lately . I haven't done much work (if any) since my last post. I have a working bare application, but not enough to even release as a beta at this point Maybe with christmas on the way, i can sqeeze in some time to release a beta and get some public feedback.

What I have so far is a working application that can browse the media files, where you can selectively find metadata for a given movie item. I need to add in the batch update capabilities and a cover art search tool, and then I'll release a beta.

Other things that I'd like to add in (after the initial release), include the ability to list files that are in the sage movie database, but not on the filesystem (ie, orphaned files). For my own personal management I also want to add in move and delete functions that can move a movie from one folder to another and update the sage database in the process. Move and deletes also need to move/delete thumbnail, properties, and subtitles as well.

Don't worry, I release something before I get to all these features, I just need to find some time to finish off a couple of little things.

[mrdrbdt]

It seems to be working really well so far. Still scanning through all couple hundred movies!

I do have one question, is it possible to download larger cover art?

[bcjenkins]

It works really well with DVD profiler. Nice cover art there.

B

[mrdrbdt]

Yeah... But having to create a DVD Profiler ... uh.. profile.. sort of defeats the "Batch Import" function. I have 1000 movies and I don't really feel like typing each one into DVD profiler. They're DIVX rips of my DVDs so DVD Profiler forces me to have to type each title in.

Oh, and I'd have to pay for DVD Profiler too...

[stuckless]

I do have one question, is it possible to download larger cover art?
Reply With Quote

Using the command line tool, there is no automated way to download better coverart. I think if you were to use google images (or some other cover art finder) and put the image url in the movie properties (_thumbnailUrl), then the tool would use that url to get the image. You will have to pass --forceThumbnail in order to have the tool overwrite an existing thumbnail.

[bcjenkins]

Originally Posted by mrdrbdt

Yeah... But having to create a DVD Profiler ... uh.. profile.. sort of defeats the "Batch Import" function. I have 1000 movies and I don't really feel like typing each one into DVD profiler. They're DIVX rips of my DVDs so DVD Profiler forces me to have to type each title in.

Oh, and I'd have to pay for DVD Profiler too...

I use a barcode scanner. $30 is not too much for well done dvd profiles and high res cover art, esp for 1000 movies.

B

[QueOnda]

Originally Posted by mrdrbdt

I do have one question, is it possible to download larger cover art?

MyMovies does a real good job in scanning a folder and getting great resolution cover art. It puts a folder.jpg in the movie folder. you need to put movie.avi in it's own folder.

That's what I have done so far. Then I was going to look at the batch IMDB Importer to just get information for me. (would it work that way?)

[stuckless]

The batch metadata importer doesn't care if the movies are in a directory or not. It will still search and download information and write to a properties file. It will, as well, download a thumbnail and save it as the same name as the movie. I'm guessing sage will show the movie folder.jpg until you enter the folder, then it will show the thumbnail associated with the movie.

[mrdrbdt]

Originally Posted by stuckless

There is a setting in the tool to force it to use ONLY a single genre entry. The default is to use more than one, which I'll change when I release another version. I misunderstood how sage uses the genre field, and since I don't use genre browsing, I didn't notice that it's not correct.

Add this your metadataupdater.properties file.

# Set to true|false to enable/disable single or multiple genre entries per line
org.jdna.media.metadata.impl.sage.SageVideoMetaData.singleGenreField=true

Is there a way to have it list two genres? That way I'll have a category and sub-category.

[stuckless]

Is there a way to have it list two genres? That way I'll have a category and sub-category.

No, but i'll look at changing the property so that you can specify how many genre levels you want instead of just a true or false.

[mrdrbdt]

I just ran the tool with the single category option set to true. Had some problems this time, it returned several errors. Here's what I got:

Listing Failed Videos....
Video: file:////mr_movie/media/Movies/%23-D/2001-%20A%20Space%20Odyssey/2001%20-%20A%20Space%20Odyssey.avi
Error: null

Video: file:////mr_movie/media/Movies/%23-D/300-HD/300-HD.avi
Error: Server returned HTTP response code: 403 for URL: http://www.imdb.com/title/tt0372006/

Video: file:////mr_movie/media/Movies/%23-D/A%20Beautiful%20Mind/A%20Beautiful%20Mind.avi
Error: null

Video: file:////mr_movie/media/Movies/%23-D/About%20Schmidt/About%20Schmidt.avi
Error: null

Video: file:////mr_movie/media/Movies/%23-D/Afgan%20Knights/Afgan%20Knights.avi
Error: Server returned HTTP response code: 403 for URL: http://www.imdb.com/title/tt0760157/

There were a few more, but just repeats of these errors on other dirs. On most of these errors, the default provider had asked me to pick the correct title, and I did. Then it would say "Failed" on the console and move on.

On the movie "300", I have two copies. One is Blu-ray and the other is DVD. The HD copy of 300 is name 300-hd.avi and the SD copy is just called 300.avi. When it scans 300-HD, it can't find a match and in the prompt where it asks me to pick a match I tried "title=300" and it still can't find a match (the options it returns are bizzare, bunch of foreign movies).

Edit: Btw, this tool is awesome. Thanks stuckless!

[stuckless]

You can download from here.

There are some significant changes with this release. Much of the code has been refactored because of the work that i've been doing with the web based metadata tool.

There is a new properties format
The property file has changed from metadataupdater.properties to metadata.properties. To see what the new properties are, see the wiki, or run

# java -jar MetadataTool.jar --showProperties

My apologies for changing the properties format, but i changed to new model that makes it easier for me to add new properties and keep the documentation in sync, and generate a commandline --showProperties that is up2date.

I've fixed the 403 error (null), and now the User Agent can be set in the properties file.

The new and updated nielm_imdb jar is now included (thanks flashbacck)

I've removed the useSingleGenre option, and now you can specify how many genre levels you want in the output (see properties, default 1)

Even though there isn't much new in this release, there are huge changes. Unfortunately, I'll be travelling next week, so if you run into problems, I won't be able to respond until late next week.

One last note.... If you have the Sage Remote APIs installed, and you are running version 6.5.8-7+, then you can use a new --refreshSageTV option to tell SageTV to refresh it's media files. This may be useful after a large scan.

[electronics4lif]

Is the metadata.properties file supposed to be included in the downloaded .zip file? I don't recall how I got the older metadataupdater.properties file.

[stuckless]

i used to ship a metadataupdater.properties.sample, which was used to show the available properties. now you can use the --showProperties to get a list of configurable properties.

When you update properties, just put the properties that you want to update in the metadata.properties.

If you want to see the list of properties in an editor, then you can do something like this..

# java -jar MetadataTool --showProperties > metadata.properties.sample

That should dump all the properties to a file.

[mrdrbdt]

The new version seems to work much better. Thanks for adding in the multi category option, it's much easier to categorize movies with two instead of one.

The only problem I've had so far is with searching for a title manually. When I get to the prompt for choosing a movie:

Originally Posted by console

Search Results: Aqua Teen Hunger Force Colon Movie for Theaters
00 (-) - Aqua Teen Hunger Force Zombie Ninja Pro-Am [2007]
01 (-) - Aqua Teen Hunger Force [2000]
LEGEND: ? Unknown; * Exact Match; - Partial Match; + Popular Match;

[q=quit, n=next (default), ##=use result ##, TITLE=Search TITLE]

I type "colon movie" to manually search for it, and I get:

Originally Posted by console

Search Results: colon movie
00 (*) - Aqua Teen Hunger Force Colon Movie Film for Theaters [2007]
LEGEND: ? Unknown; * Exact Match; - Partial Match; + Popular Match;


[q=quit, n=next (default), ##=use result ##, TITLE=Search TITLE]

I then type "00" and get this:

Originally Posted by console

Updated: Aqua Teen Hunger Force Colon Movie Film for Theaters (Aqua Teen Hunger Force - Colon Movie for Theaters.avi
)
Updated: Aqua Teen Hunger Force Zombie Ninja Pro-Am (Aqua Teen Hunger Force - Colon Movie for Theaters.avi)

Looking at the properties file, it looks like it is actually updated with the second movie, which isn't correct.

I looked at the metaupdated logs, and it looks like eveything works right. If you'd like the logs, let me know.

[stuckless]

It's a bug. I just uploaded a 1.9 version, which should fix the issue.

1.9 uses sagex-api-6.4.8-9 of the remote apis. If you are going to use the --refreshSageTV option, then you'll need the updated version on the server as well. If you are using Jetty, then grab the updated sagex.xml from the wiki as well.