Utility: Batch Metadata Importer 3.x

[plnewton]

Duh!! the log file is the stream of data that has been flashing before me the last 2 hours

It seems to fail here:
Skipping: file:/m:/movielibrary/You%20Don't%20Mess%20with%20the%20Zohan/; Messag
e: Failed to find/update metadata for resource: file:/m:/movielibrary/You%20Don'
t%20Mess%20with%20the%20Zohan/
10713042 [main] DEBUG sagex.phoenix.configuration.ConfigurationManager - Gettin
g Scoped Property: CLIENT; for key: bmt/metadataUpdater/rememberSelectedSearches

10713042 [main] DEBUG org.jdna.metadataupdater.BMTSageAPIProvider - GetProperty
; Key: bmt/metadataUpdater/rememberSelectedSearches; Value: null
10713042 [main] DEBUG sagex.phoenix.configuration.ConfigurationManager - Settin
g Scoped Property: CLIENT; for Key: bmt/metadataUpdater/rememberSelectedSearches
; with value: true
10713044 [main] DEBUG org.jdna.metadataupdater.BMTSageAPIProvider - SetProperty
; Key: bmt/metadataUpdater/rememberSelectedSearches; Value: true
10713044 [main] DEBUG org.jdna.configuration.ConfigurationManager - Writing Tit
le mappings: C:\Program Files\SageTV\SageTV\metadata-titles.properties
10713046 [main] ERROR org.jdna.metadataupdater.MetadataUpdater - Failed to proc
ess Video MetaData!
java.io.FileNotFoundException: metadata-titles.properties (Access is denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at org.jdna.util.PropertiesUtils.store(PropertiesUtils.java:35)
at org.jdna.configuration.ConfigurationManager.saveTitleMappings(Configu
rationManager.java:59)
at org.jdna.metadataupdater.MetadataUpdater.process(MetadataUpdater.java
:338)
at org.jdna.metadataupdater.MetadataUpdater.run(MetadataUpdater.java:136
)
at org.jdna.metadataupdater.MetadataUpdater.main(MetadataUpdater.java:91
)
Processing Failed, see log for details.

Sorry, not knowing it well enough yet to work out what the problem is here???

[stuckless]

Originally Posted by plnewton

Duh!! the log file is the stream of data that has been flashing before me the last 2 hours

It seems to fail here:
Skipping: file:/m:/movielibrary/You%20Don't%20Mess%20with%20the%20Zohan/; Messag
e: Failed to find/update metadata for resource: file:/m:/movielibrary/You%20Don'
t%20Mess%20with%20the%20Zohan/
10713042 [main] DEBUG sagex.phoenix.configuration.ConfigurationManager - Gettin
g Scoped Property: CLIENT; for key: bmt/metadataUpdater/rememberSelectedSearches

10713042 [main] DEBUG org.jdna.metadataupdater.BMTSageAPIProvider - GetProperty
; Key: bmt/metadataUpdater/rememberSelectedSearches; Value: null
10713042 [main] DEBUG sagex.phoenix.configuration.ConfigurationManager - Settin
g Scoped Property: CLIENT; for Key: bmt/metadataUpdater/rememberSelectedSearches
; with value: true
10713044 [main] DEBUG org.jdna.metadataupdater.BMTSageAPIProvider - SetProperty
; Key: bmt/metadataUpdater/rememberSelectedSearches; Value: true
10713044 [main] DEBUG org.jdna.configuration.ConfigurationManager - Writing Tit
le mappings: C:\Program Files\SageTV\SageTV\metadata-titles.properties
10713046 [main] ERROR org.jdna.metadataupdater.MetadataUpdater - Failed to proc
ess Video MetaData!
java.io.FileNotFoundException: metadata-titles.properties (Access is denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at org.jdna.util.PropertiesUtils.store(PropertiesUtils.java:35)
at org.jdna.configuration.ConfigurationManager.saveTitleMappings(Configu
rationManager.java:59)
at org.jdna.metadataupdater.MetadataUpdater.process(MetadataUpdater.java
:338)
at org.jdna.metadataupdater.MetadataUpdater.run(MetadataUpdater.java:136
)
at org.jdna.metadataupdater.MetadataUpdater.main(MetadataUpdater.java:91
)
Processing Failed, see log for details.

Sorry, not knowing it well enough yet to work out what the problem is here???

It would seem that you found the logs... but for future reference, I there is a "Configure Logging" link in the first post. Also, in 3.0, you can configure the logging from the web UI, using the log4j configuration panel.

And.... for someone that had no idea what you were looking for, I think that you found the error

From what I can tell by looking at this error, is that BMT failed because it could not write to the metadata-titles.properties file. This file is used to record your "selections" when you are prompted for a movie selection. It does this, so if a rescan is required, and --auto=true, then it will check this file for your selection, instead of prompting you. In your case, it could not create/write to the file.

When you are running commandline, are you running it from the SageTV home dir in the Program Files area? If so, then are you running it as the same user that the SageTV service is running? If not, then that may be your problem.

Also, if you are running vista with UAC turned on, then running the command line as a different user than the sagetv server may cause other issues as well. ie, the metadata-titles.properties that is created as the commandline user may not be visible to sagetv when you are using the on-demand searches, or automatic plugin.

Just out of curiousity, which version are you testing with now, 2.x or 3.0?

[bluenote]

Originally Posted by stuckless

What SageMC version are you running?
What BMT version are you running?
What SageTV version are you running?

When you installed it, did you install sagemc, and then install the bmt package?

Are you testing this from a client or placeshifter? if a client, do you have bmt installed and configured on the client?

I installed the latest sage beta from two days ago, (file labeled 6 5 20 setup)
then sagemc from the sticky in this forum (6 3 9 a 46),
and then metadata updater 2.3.1 .

Im using sage in service mode, so , the only place I installed it was on the "client" that lives on that box.

Thanks for any help you can provide. Im feeling my way in the dark for the most part because I don't know what markers to be looking for.

thanks

Cory

EDIT: whups, I just realized there are betas that arent referenced in the 'downloads' link from the first page, just from the releases thread. Im guessing that thats my problem. I'll try now.

EDIT EDIT: Success! This is an *awesome* plugin, thanks stuckless.

[plnewton]

Hi Stuckless,

I am running Version 3 now on Windows 7 actually.

BMT is creating .properties files in my m:\movielibrary folder for each movie title but it does not seem to correspond with where the scan was up to when it failed. It seems to be writing all the .properties files ok

Should it be creating a file called You Don't Mess with the Zohan.properties
and is it failing at this point?


Cheers!!

[plnewton]

Also I have just noticed after I have refreshed the SageTV Media Centre
it is now only showing 152 titles, not sure if this is related or not???


Sorry, one more question, is there a command to scan for this one movie so I can test with rather than do the entire folder?

[stuckless]

Originally Posted by plnewton

Hi Stuckless,

I am running Version 3 now on Windows 7 actually.

BMT is creating .properties files in my m:\movielibrary folder for each movie title but it does not seem to correspond with where the scan was up to when it failed. It seems to be writing all the .properties files ok

Should it be creating a file called You Don't Mess with the Zohan.properties
and is it failing at this point?

Cheers!!

It would appear to me that it did not find a match for "You Don't Mess with the Zohan" and as a result, it tried to read metadata-titles.properties to see if there was a match in there. But it failed to read that file (and create it if does not exist) and it failed.

So while the metadata properties are stored in your M:\movielibrary, bmt configuration files are stored in the same directory as the tool. ie, the same directory where MetadataTools.jar is located, and it's in that directory where bmt will try to write some extra files, like the log file, metadata-titles.properties, etc. If you were seeing log information in your command line console, then that's probably because the tool did not have the permissions to create/write to a log file in that same directory.

As a rule of thumb, you'll want to run the command line tool as the same user as the SageTV server process, IF you are running it from the SageTV directory.

But, you can also extract the tool to ANY arbitrary directory and run it as a command line tool from that directory that has write access by your user.

I typically just extract it to my SageTV area, since I run it as a plugin, but when I run it from the commandline, I make sure that I run it as the same user as my SageTV server process, so that I don't get permission problems.

EDIT: I don't know why it would have failed to find a match on that movie. I just tried that movie, and I had no issues doing a lookup. I suspect that there was some error before the metadata-titles.properties error that may shed some light on things as well.

[stuckless]

Originally Posted by plnewton

Also I have just noticed after I have refreshed the SageTV Media Centre
it is now only showing 152 titles, not sure if this is related or not???


Sorry, one more question, is there a command to scan for this one movie so I can test with rather than do the entire folder?

How many movies should it have? And several people have noticed that for some unknown reason, their library will not reflect the total # of movies if the automatic plugin is installed. I'm currently trying to troubleshoot this issue. From the webui, does it show that it's still doing a Library Import Scan? If so, then Sage may not have completed the scan and as a result, it may be showing the total # of movies.

From the command line, you can pass a movie or a folder. if it's a dvd folder, then just pass the dvd folder as the movie.

ie,

# java  -jar MetadataTool.jar  "m:\movies\Don't mess with the zohan\"

Obviously you'll need you other command line options as well, but the last parameter that you pass to the command is a movie file, dvd folder, or root movie folder.

I think a few posts back, that razrsharpe posted a bat file that you could use for dragging a folder/file onto the bmt command, i think. I don't use windows, so I'm not sure about the exact usage of the bat file. I should update the first post with a link to that file, so that windows users can use it, if needed.

[plnewton]

Originally Posted by stuckless

How many movies should it have? And several people have noticed that for some unknown reason, their library will not reflect the total # of movies if the automatic plugin is installed. I'm currently trying to troubleshoot this issue. From the webui, does it show that it's still doing a Library Import Scan? If so, then Sage may not have completed the scan and as a result, it may be showing the total # of movies.

It is showing a 128 out of 329 that are in that folder. there is also another folder in SageTV which has about 24 titles in it and they are all showing up fine
So in to total it is showing 152 titles

[stuckless]

I can't explain why it would do this... but babgvant sent me some information that may prove to be a work around for this problem as well. I'll try to get another release out this weekend.

[plnewton]

OK, look forward to the update..

Seem to have got passed the processing failed problem, you were right it is permissions related.

I am running through the list again to see how it goes

[plnewton]

Just on the issue of it no longer showing all the titles..

It seems that modifying the Import Directories has no effect any more either.

I have another folder where I keep some home movies which is included in the import Directories, I have removed this from the SageTV MC and only have the movielibrary folder showing up but after a refresh it still shows all the home movies

[benjamintm]

I'm trying to use BMT to catalog my TV series. Most work correctly, but a couple have given me headaches, such as Firefly and Wonderfalls. The .properties created for the episodes are incorrect (for the DVD order, not the aired order). For Firefly I was able to copy and paste to correct the files, but with Wonderfalls Sage refuses to update even with the properties set correctly. In otherwords, Episode 2 in sage is still pointing to the episode 3 file, even with the .properties file being set correctly. Is this a caching problem? Is Sage not re-reading the correct .properties file?

Thanks,
Ben

[stuckless]

Originally Posted by benjamintm

I'm trying to use BMT to catalog my TV series. Most work correctly, but a couple have given me headaches, such as Firefly and Wonderfalls. The .properties created for the episodes are incorrect (for the DVD order, not the aired order). For Firefly I was able to copy and paste to correct the files, but with Wonderfalls Sage refuses to update even with the properties set correctly. In otherwords, Episode 2 in sage is still pointing to the episode 3 file, even with the .properties file being set correctly. Is this a caching problem? Is Sage not re-reading the correct .properties file?

Thanks,
Ben

Ben, Sage will only reread the .properties if the timestamp on the media file is updated since it was last imported. Sage uses this a reasonable test so that it doesn't have to keep polling for property changes.

I don't think windows has any "touch" command (at least I can't find it), so, if you are running windows, then I think your options are....

1. Remove the Movie from your sage library. Then tell sage to rescan for new media (it will detect that the movie is gone). Then copy the movie back in along with its .properties file and then tell sage to rescan for new media (should pick up your movie and metadata) (if you are savy enough, then can see sage removing and adding your mediafile by looking in the sagetv_0.txt log file, if you have logging enabled)

2. Using some tool, update the time stamp on the mediafile. If it's a DVD folder, then I think you'll need to update the timestamp on the VIDEO_TS folder.

good luck

[babgvant]

Originally Posted by stuckless

I don't think windows has any "touch" command (at least I can't find it), so, if you are running windows, then I think your options are....

There isn't a OOTB version, but a 3rd party version can be found here.

For the command line challenged, copy this into a .reg file (changing the path to touch of course, but make sure to keep the \ = \\ syntax in the path), run it and a "touch" option will show up in the right-click menu.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\*\shell]

[HKEY_CLASSES_ROOT\*\shell\touch]

[HKEY_CLASSES_ROOT\*\shell\touch\command]
@="\"E:\\Data\\Documents\\Tools\\touch\\touch.exe\" \"%1\""

EDIT: attached touch.exe; forgot that you need a codeproject account to dl from there.

[jphipps]

I am having a problem with the library import for certain movies. I recieve the following messages in the debuglog:

Sun 7/12 11:58:39.663 [LibraryImporter@17506c0] Seeker is starting the library import scan...
Sun 7/12 11:58:39.683 [LibraryImporter@17506c0] Starting to scan lib import root: \\livingroom\data\Movies
Sun 7/12 11:58:43.029 [LibraryImporter@17506c0] BMT: Processed File: \\livingroom\data\Movies\Family\Backyardigans\BACKYARDIGANS INTO THE DEEP DOM\VIDEO_TS
Sun 7/12 11:58:43.033 [LibraryImporter@17506c0] Seeker has finished the library import scan.
Sun 7/12 11:58:43.033 [LibraryImporter@17506c0] PooledThread ended w/ an exception: java.lang.NullPointerException

If I remove the BMT metadata plugin, the import works. I have also attached a debug log from bmt. The times will not match up, because between restarting sage, I lost the debug log for the same time period as the bmt debug log, but it was the same messages..

Let me know if you need any other information.

Thanks,
Jeff

[sflamm]

Install CygWin:

http://www.cygwin.com/

Or

Install the Windows Powershell:

http://www.microsoft.com/windowsserv.../download.mspx

Use the script found here:

http://devcentral.f5.com/weblogs/Joe...l---touch.aspx

[MeInMaui]

Originally Posted by babgvant

There isn't a OOTB version, but a 3rd party version can be found here.

For the command line challenged, copy this into a .reg file (changing the path to touch of course, but make sure to keep the \ = \\ syntax in the path), run it and a "touch" option will show up in the right-click menu.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\*\shell]

[HKEY_CLASSES_ROOT\*\shell\touch]

[HKEY_CLASSES_ROOT\*\shell\touch\command]
@="\"E:\\Data\\Documents\\Tools\\touch\\touch.exe\" \"%1\""

This is the coolest tip I've seen in a while. I'll definitely have to set this up tonight. (I'm not command line challenged, but a context menu item sure is convenient) Thanks Andy!

Aloha,
Mike

[stuckless]

Originally Posted by jphipps

I am having a problem with the library import for certain movies. I recieve the following messages in the debuglog:

Sun 7/12 11:58:39.663 [LibraryImporter@17506c0] Seeker is starting the library import scan...
Sun 7/12 11:58:39.683 [LibraryImporter@17506c0] Starting to scan lib import root: \\livingroom\data\Movies
Sun 7/12 11:58:43.029 [LibraryImporter@17506c0] BMT: Processed File: \\livingroom\data\Movies\Family\Backyardigans\BACKYARDIGANS INTO THE DEEP DOM\VIDEO_TS
Sun 7/12 11:58:43.033 [LibraryImporter@17506c0] Seeker has finished the library import scan.
Sun 7/12 11:58:43.033 [LibraryImporter@17506c0] PooledThread ended w/ an exception: java.lang.NullPointerException

If I remove the BMT metadata plugin, the import works. I have also attached a debug log from bmt. The times will not match up, because between restarting sage, I lost the debug log for the same time period as the bmt debug log, but it was the same messages..

Let me know if you need any other information.

Thanks,
Jeff

Jeff, try the beta8 with the "workaround" enabled (see the release notes in the releases thread). I think this is related. This exception happens outside of the bmt apis, but it's definately related to what bmt is passing back. The work around, basically returns null instead of the metadata, and then sage reads the metadata directly from the .properties, so it's somewhat transparent from a user perspective.

[stuckless]

I've updated the first post to include the tips by babgvant (very slick) and sflamm's links. Also, as for the latest beta8, i've decided to implement a --touch feature in the commandline.

[babgvant]

Originally Posted by stuckless

I've updated the first post to include the tips by babgvant (very slick)

I can't claim credit, it was "borrowed" from the comments for the tool.