Utility: Batch Metadata Importer 3.x

[rm1759]

Thank you very much for this tool, it is really good!

So I installed it, and got it working, very painless.

However, now I am having an issue getting sage to read the properties file.

My structure (windows XP) is <drive>:\Movies\Movie_Name\video_ts

When I ran the tool, it generated all of the .properties and downloaded the .jpg cover art. I used the --recursive switch. (Unfortunately, it put the properties and jpg in the root <drive>:\Movies, and not under the movie name dir, not a big deal, but it would be nice not to have to move the files around afterwards, I have about 600 movies on my drive(s))

After running the tool, I copied the prop and jpg onto <drive>:\movie\movie_name directory, selected re-import library, that did not work, then I also copied it into the video_ts directory, re-imported, and still the data is not showing up in sage. am I missing something simple?

[stuckless]

It was my understanding, from the previous posts, that the .jpg and .properties need to be at the same level as the movie folder... Maybe I'm still missing.... I should actually copy a dvd locally and test to make it's doing what it should be doing

[rm1759]

that was what I thought too.

Using the -recurse switch though, the files were all placed at the <drive>:\movies folder, not in the <drive>:\movies\movie_name folder...

Sage did not pick up the properties file in the movies dir either...

is there any switch in sage to tell it to use the properties file?

[JREkiwi]

Originally Posted by stuckless

It was my understanding, from the previous posts, that the .jpg and .properties need to be at the same level as the movie folder...

That's correct.

SageTV only reads the .properties files when a video is initially added to the DB. (It will pick up a .jpg on a manual rescan) On videos you can "trick" it to do that by changing the time stamp of the file. I've not done any testing as to what files(s) need to change in a DVD structure in order for it to see it as changed and rescan (resulting in the .properties being picked up).

John

[rm1759]

Originally Posted by JREkiwi

That's correct.

SageTV only reads the .properties files when a video is initially added to the DB. (It will pick up a .jpg on a manual rescan) On videos you can "trick" it to do that by changing the time stamp of the file. I've not done any testing as to what files(s) need to change in a DVD structure in order for it to see it as changed and rescan (resulting in the .properties being picked up).

John

Ahh, that must be the trick.

I imagine if I remove my import directory, and then re-add it, that would work as well.

The .jpg though did not seem to be picking up. or at least it was not displaying the cover art on the videos screen on the HD-100 (or on the sage server)

[JREkiwi]

Originally Posted by rm1759

I imagine if I remove my import directory, and then re-add it, that would work as well.

Yes

Originally Posted by rm1759

The .jpg though did not seem to be picking up. or at least it was not displaying the cover art on the videos screen on the HD-100 (or on the sage server)

You need to do a manual "Refresh Imported Media" to get the .jpg picked up. Or, removing, re-adding the import directory will work with that too. Also SageTV does cache the images as well, so if there is already an image in the cache, you need to restart SageTV to display the new image.

John

[stuckless]

Currently when the utility runs, it will "touch" the file on which metadata is being created. For dvd folders, it's the actualy directory. Maybe I need to change it to touch all files in the directory, and just the directory itself.

[jusjoken]

FYI... Sage replied to my bug report regarding importing the genre field from a .properties file and is fixing this in the next version... see my post here...

http://forums.sagetv.com/forums/show...7&postcount=13

[rm1759]

OK, now I am feeling very silly.

I removed the import directory. refreshed the library. Then waited (seemed like it takes a long time to refresh the library > 10 minutes, is there a way to see the current status?).

After all of the movies from my test directory were removed (~300 of them) i added the dir back into sage and then waited again. Once the movies finally came back, I still have no metadata.

the .jpg and .properties files are stored at the root of the movies dir (the dir sage is pointed to) there is a directory for each movie, with video_ts dir under them.

on one movie I tried putting the files in the movie_name dir, and video_ts dir (copies of the same 2 files in each) and on another I just put the 2 files in the movie_name folder.

neither of these is showing the data.

does the metadata load at the same time as the movies? it has been 30 minutes or so since I selected the refresh library...

[stuckless]

Sorry I don't have any DVD folders, so I wrote that part of the code blind. I do know that there is a potential bug in that I was updating the timestamp on the dvd folder, and NOT on the actual files (i've updated the code, but I have not re-released). So this may be a contributing factor, but if you've removed and re-added your movies, then this should not affect it.

As a test, I would remove your movies, and just add in a single dvd movie and see what happens. Sage should refresh pretty quickly. I have over 300 movies (in avi format) and they completely refresh in minutes.

[rm1759]

I think I have figured out my problem.

I am on the current prod. version (6.3). The latest beta is 6.4.5. In this beta, they added support for metadata on DVD files. I am downloading this beta now and I will update after I install it...

[stuckless]

I've created a new version 1.3-test in the download area

In this version I added --offline for offline video support, cached urls, some minor dvd folder changes, and a dvd profiler implemenation for fetching metadata.

If you want to try the dvd profiler implemenation, then be sure to put your dvd profiler url in a custom property file, and pass the --provider=dvdprofiler on the command line.

[jusjoken]

Thanks for the new version. I tested general functionality from the old version and everything seems to still work good.

I tested the --offline function and it worked great... except... if the --offline flag is set, it not only creates the specific movie files (jpg,properties and avi) but it also "touches" every file in the directory including subdirectories (no --recurse set)... by "touches" I mean every file in the directory has a new timestamp.

I tested the --provider=dvdprofiler and I could not get any results for any movie that I have in my collection. I included my .log and .properties file in the attached zip. Let me know if you need anything else to find the issue.

Ken

[stuckless]

I'll check this out later.... Funny thing about dvd profiler... I built it using 1 url and then I tested on a second... and it failed So i made the changes to make it work on the second url. I have a feeling that each dvd profiler url is going to be slightly different, and if that's the case, I may have drop support for it, since I can't possibly account for everything I'll check out your url tonight and see what happens.

Thanks.

[jusjoken]

The approach for DVD Profiler support within SageMC is slightly different. The path to a local directory containing the images (basically the path to the DVD Profiler application store) and another path for the collection.xml file exported from DVD Profiler are stored in the sage properties file and then SageMC gets the moving info from the XML file and using that info can then get the jpg files from the application storage location.

Just a thought to a different approach if the url approach doesn't work.

Ken

[stuckless]

Just a thought to a different approach if the url approach doesn't work.

I actually don't have dvd profiler (nor do I run windows for that matter), but if someone want's to post the directory structure of dvdprofiler, xml, and jpgs, then I'd consider writing a local version of dvd profiler.

I Used the URL approach because you can specify as many urls as you like, and it indexes all of them (using lucene). So if you had enough urls, then you'd effectively would not need to manage your own collection, but let others do it for you

[jusjoken]

Sample DVD Profiler structure in attached zip file. I included just a handfull of movies as an example.

Notes: DVD Profiler lets you store your "database" anywhere. So the path in the zip included is relative to some path that would need to be in your properties file. Basically you just need a path to where the "images" directory is stored. There are usually other files in the same level as the images directory but these are DVD Profiler database files so I did not include them. The collection.xml file is exported in a seperate function in the application, and during export you are asked where to save the xml file, so it could be placed anywhere. So I would suggest 2 paths in the properties, 1 for the images directory, and 1 for the location of the collection.xml file.

If you want a tester, just let me know.

[stuckless]

New Release 1.3.1

See Release Notes, but basically, nothing new, just fixed some issues with dvd profiler and --offline.

Thanks Ken for the DVD Profiler zip. I'll take a look at it, but my weekend is currently booked... I just got my new HD-PVR delivered to my door

[jusjoken]

I found the same issues... all files in the directory get touched if the --offline flag is true and the DVD Profiler provider returns nothing.

Log file for the DVD Profiler test attached (properties file was untouched from my last tests)

[stuckless]

Try downloading again... make sure it's the 1.3.1 version (the download link should serve it up.) If you have doubts, then download it directly from the google site.

I just downloaded the package and ran in a clean dir, and it worked fine. I can tell from the log that it is not the 1.3.1 version, since I removed the log lines, "Target: null", since they were not very imformative if you are downloading and extracting over a previous version, make sure that it is actually overwriting the old version. Also, delete the cache directory if it exists.

hope this helps...