Utility: Batch Metadata Importer 3.x

[mkanet]

This has happened to me too with BMT. However, I haven' tried BMT 3.1 yet. I'm currently using 3.03. I have seen this kind of an issue on other plugins too; with the settings not being able to be changed. Another related symptom is the options menu being distorted with overlapped text and border lines. Sean is right, this does happen with the same plugin loaded on top of another version of it (as this has happened to me with another plugin).

You can try to load your STV (not a custom one) and import BMT only.. then try changing the setting again. If it works, then keep adding imports until it stops working.

I dont think I've ever been able to successfully change settings through the BMT configuration menu. However, I can make the settings changes through jetty.

Originally Posted by gplasky

OK. It looks like I was wrong. I just THOUGHT I updated to 3.1 Turns out it was 3.03. Sorry about that. ANd that explains most of it.

My current issue is on a PC Client.


I can't enable the metadata configuration after importing either the STVI in the default or the xml in SageMC. This was working last week. I've looked at it a couple of times now, backed up and did a clean install with ONLY the metadata importer being the only plugin. It says "Disabled" and when I click it or push "enter" it doesn't change. I've made sure none of the files are read-only now. The HD100 and HD200 and the WHS server plugins are fine and can be changed with no issues. My next step is to roll back java unless I'm missing something obvious.

Gerry

[gplasky]

Originally Posted by stuckless

Gerry, I've looked at the logs, and I can see this...

Sat 9/12 15:14:56.794 [AWT-EventQueue-0@193c0cf] EXCEPTION in getValueAsObject:sage.a.e: Error in method reflection of bmt_api_IsBMTPluginInstalled of java.lang.reflect.InvocationTargetException for:If(bmt_api_IsBMTPluginInstalled(), "Enabled", "Disabled")

Which is the root of your issue. Unfortunately I don't know why that bmt api call would be throwing an exception... so i'll run down some possible scenarios.

1. You are running the stv plugin on a client, but you haven't installed the bmt plugin on the client machine. (bmt has to be extraced on both the server and client machines)
2. You installed the bmt plugin over an existing plugin, but you didn't "overwrite" the previous jars.
3. You are running vista and somehow UAC is messing with the files
4. something else

Thanks. I'm actually on Windows 7 RTM Ultimate but I've turned off UAC from the get go. But I was also beta testing a security app so that is probably the root cause. I'll let you know what I find out.

Gerry

[r00st3r]

Thanks for the verbose detail. Unfortunately, you are not alone. Other people seem to be experiencing the same issue. I don't understand why the File System view in SageMC would not correctly show the File System layout. Have you tried the filesystem view in the default stv to see if it also is affected?

I switch to the default STV and it organizes them fine in Folder mode. When I switch back to SageMC, it does not.

[stuckless]

Ok, i'll follow up with someone on the SageMC side to see how bmt has caused the filesystem view to become broken.

[MeInMaui]

There could be some peculiarity in the SageMC code for the folder view. I didn't program that section so I don't have any insights off the top of my head. I'll have to try to take a look when I have some time. I'll work with Stuckless on this.

Aloha,
Mike

[wayner]

Can I use the web interface of BMT to manually edit metadata? I thought I could but this doesn't seem to work. I go into the metadata, make some changes and push Save and it then regenerates back to what I started with. Am I doing something wrong?

[r00st3r]

Originally Posted by MeInMaui

There could be some peculiarity in the SageMC code for the folder view. I didn't program that section so I don't have any insights off the top of my head. I'll have to try to take a look when I have some time. I'll work with Stuckless on this.

Aloha,
Mike

Thanks Maui and Stuckless. I know you probably have better things to do.
In the meantime, I wrote a batch script to remove the .properties file from all my media and I am doing a full library scan. I don't know if this will work or not, but it seems it's how it first started.

[mkanet]

After upgrading from BMT 3.03 to BMT 3.1, the automatic plugin works fine. However, the directory sageTV records TV to is completely ignored. All the folders/subfolders that are defined in Jetty/SageTV/Video Source will be scanned and correctly download metadata and fanart.

I tried to add my TV recording directory in under Jetty/SageTV/Video Source, but it will not add that folder.

I didnt have this issue in 3.03. However, I didnt try 3.02 or 3.01.

Maybe this makes more sense to you Sean. I have no idea why my TV recordings folder(s) are ignored for the automatic plugin.

EDIT: I removed my video imports folders so it will be easier to isolate the problem. After doing this and running a full rescan of everything, 0 files get scanned and mailcheck does not get created this time. Mailcheck only gets created if I add the video import folders back.

I already have everything set to "debug" instead of "info".

This is the same machine that the TV recordings worked for earlier today when I had version 3.03 installed. Unfortunately, I dont have version 3.03 or 3.02 to revert back to. Maybe someone can give me a good copy?

[stuckless]

Originally Posted by mkanet

After upgrading from BMT 3.03 to BMT 3.1, the automatic plugin works fine. However, the directory sageTV records TV to is completely ignored.

I don't know why that would be the case. In my setup, my recordings dir is processed. If you use the browse feature and only select "Recordings", do you get any results?

PS: Can someone please post BMT 3.02 and 3.03? I cant find those anywhere; not even on the Wiki/project site.

All old downloads can be found in the deprecated downloads section.

[mkanet]

Okay, we're getting somewhere now. Using the browse feature, I can see all the metadata. I can see where the fanart folders used to be... no fanart files!. The fanart files are missing because I deleted them and wanted to get BMT to reopopulate the files to make sure that the full rescan still works.

Maybe this is by design, and BMT will not download/re-add fanart that has been downloaded before; EVEN when automatic plugin is set to overwrite fanart/metadata.

Sean, could you tell me how to redownload fanart files that I erased? I'm guessing its confused because the metadata is still there for the TV files.... however the jpg's are not there since I erased them.

EDIT: SUCCESS! I just used the scan option and selected to overwrite existing fanart/metadata. For some reason, this works, however, the options under automatic/overwrite fanart/metadata dont work.

EDIT2: I'm not sure you know this or not, but you have a small typo in BMT 3.1 Brose.

Originally Posted by stuckless

I don't know why that would be the case. In my setup, my recordings dir is processed. If you use the browse feature and only select "Recordings", do you get any results?



All old downloads can be found in the deprecated downloads section.

[Fuzzy]

Originally Posted by stuckless

Could you post a filename special angent oso and I'll give it try and see what's up. As for the &/And issue... i'm not sure what to do about this. I don't want to blindly replace & and And or vice versa, because that's just going to cause me headaches in other areas It almost needs it's own special configuration file that will lookup a title that will override the current title for a given show... either way, not exactly trivial. I'll add a enhancement request... and I'll see what I can come up with.

Example Special Agent Oso episode:
SpecialAgentOso-DrOffLicensetoDress-2089020-0.mpg

As for the &, basically, I'm looking for a way to have BMT find/download metadata, regardless of whether it is showing AND or &, but NOT change the title. I don't like the title being changed, as I mentioned, because it loses favorite association. This has multiple problems, such as auto-delete settings, keep-up-to requirements and such.

[stuckless]

Originally Posted by mkanet

EDIT2: I'm not sure you know this or not, but you have a small typo in BMT 3.1 Brose.

It's sad... but I only just noticed that about an hour before you posted it A classic example of my brain seeing what I want it to see

As for the overwrite flags for the plugin, they should work, but I haven't tested them explicitly for a while. I'll take a quick look and see if they are still be honoured.

[stuckless]

Originally Posted by Fuzzy

Example Special Agent Oso episode:
SpecialAgentOso-DrOffLicensetoDress-2089020-0.mpg

I didn't test this from within the sage application, but I did quickly test it from the command line using...

java -jar MetadataTool.jar --offline --metadataOnly "SpecialAgentOso-DrOffLicensetoDress-2089020-0.mpg"

and it did lookup the file and find it correctly... the .properties were...

#Generator: Batch Metadata Tools (3.1); MediaFile: file:/home/seans/DevelopmentProjects/workspaces/sage/MovieMetadataUpdater/target/sage/SpecialAgentOso-DrOffLicensetoDress-2089020-0.mpg
#Sun Sep 13 08:38:11 EDT 2009
Description=Dr. Off\: A special alert is called as Oso searches for a hidden tunnel in the city. He teaches a girl, Adison, how to conserve power and water. They must turn off the unused lights, TVs and water faucets before family game night. License to Dress\: While Oso dresses into his scuba gear for a training assignment, he receives a special alert. He helps a boy, Frank, dress for school and Oso learns to dress by himself as well. Can Oso and Frank get dressed properly before Frank leaves for school?
EpisodeNumber=11
EpisodeTitle=Dr. Off; License to Dress
Genre=Animation
MediaProviderDataID=tvdb\:112931
MediaTitle=Special Agent Oso
MediaType=TV
OriginalAirDate=2009-04-18
Rated=TV-Y
RunningTime=1500000
SeasonNumber=1
Title=Special Agent Oso - S01E11 - Dr. Off; License to Dress
Year=2009
x-MetadataProviderDataUrl=112931;;;;EPISODE_TITLE\=Dr+Off+Licenseto+Dress
x-MetadataProviderId=tvdb
x-ser-Genres=Animation;Children

As for the &, basically, I'm looking for a way to have BMT find/download metadata, regardless of whether it is showing AND or &, but NOT change the title. I don't like the title being changed, as I mentioned, because it loses favorite association. This has multiple problems, such as auto-delete settings, keep-up-to requirements and such.

This is actually 2 separate enhancements... the first, would result in 1 or more extra searches per movie, per provider. (search for title as is, and then search for title replacing & with AND and then again with AND replaced with &)

The second part of NOT change the title, is a separate issue, and somewhat more complex. Fundamentally, bmt does change the display title for all shows to match a mask that you want for each media type (tv, recording, movie, multidisc,etc). To implement this, I'd need to add a flag that says to NOT change the title for all scanned items.... or at the very least for all scanned items of a given type.

[razrsharpe]

Originally Posted by stuckless

The second part of NOT change the title, is a separate issue, and somewhat more complex. Fundamentally, bmt does change the display title for all shows to match a mask that you want for each media type (tv, recording, movie, multidisc,etc). To implement this, I'd need to add a flag that says to NOT change the title for all scanned items.... or at the very least for all scanned items of a given type.

or have a lookup table similar to your title.properties file that says if sage says the title is x, match it with a particulat thetvdb show but do not change the title... this would also fix thetvdb's habit of putting year names in the show title if its a remake ( ie - cupid (2009) )...

[KJake]

FYI, BMT started dropping dash/hyphen at some point and won't find "Stargate SG-1" automatically. If I try from the command line, I am prompted for every video - not fun for 10 seasons.

[razrsharpe]

Sean,
First i gotta say i love 3.1. BMT is really becoming quite polished, thanks It's working great for me so far...

I have couple questions/requests about the browse.

• Is it possible for the input boxes for the longer strings to be longer and/or multiple lines... I'm thinking specifically about the Description and Episode/Show Title.
• What does "Show Title" display?
• Is the browse feature supposed to display the info in the sagedb or just what is in the properties file? On my setup, sage recordings that have not been run through BMT (They do have the default sage generated EPG metadata) it seems to be displaying the description and not much else. It would be awesome if the browse showed all of the metadata for a given item, similar to how nielms webserver does.... then BMT can be the main interface to view and edit ALL of the metadata (sage generated, bmt generated, default and extra metadata fields)... See the attached jpgs for the differences between what bmt and the webserver shows me. It's entirely possible i missed the boat here... if so just let me know ...

[r00st3r]

Ok, i'll follow up with someone on the SageMC side to see how bmt has caused the filesystem view to become broken.

There could be some peculiarity in the SageMC code for the folder view. I didn't program that section so I don't have any insights off the top of my head. I'll have to try to take a look when I have some time. I'll work with Stuckless on this.

Here is what I did to fix my File System menu mashup and it seems to work in my situation YMMV.

• I Removed all files (.properties, .jpg. etc..) except the .avi files in my video directories in windows using a batch file. Only the video files remain in each folder.
• Removed the import folder from within Sage.
• Stopped and restarted Sage.
• Waited until the video section in Sage was empty. No files listed.
• Re-added the import folder and did a full scan. Everything was back in it's proper place with metadata from BMT.

It appears to work fine when BMT automatically gets the metadata files. When I used the web gui to update one show, it appears in Sage at the root level of the videos section instead of it's relative File System location. Which is how this all started.

I don't know if this makes sense, or helps pin point where the issue is. I think the problem is when the web gui is used to get the metadata, it breaks the File System listings in SAGE MC.

[stuckless]

Originally Posted by razrsharpe

or have a lookup table similar to your title.properties file that says if sage says the title is x, match it with a particulat thetvdb show but do not change the title... this would also fix thetvdb's habit of putting year names in the show title if its a remake ( ie - cupid (2009) )...

I think that's the route that I'd like to take, but instead of having a separate file, i think what I'd do is have a single file (probably xml) that will allow you to set the provider information, and specific properties, such as Title, etc. I'd also provider a web ui for editing that information.

[stuckless]

Originally Posted by KJake

FYI, BMT started dropping dash/hyphen at some point and won't find "Stargate SG-1" automatically. If I try from the command line, I am prompted for every video - not fun for 10 seasons.

Could you post a sample filename that I can use for testing. it should have prompted for the first time, but then it should have stored that selection and never prompt again.

[stuckless]

Originally Posted by razrsharpe

Sean,
I have couple questions/requests about the browse.

Is it possible for the input boxes for the longer strings to be longer and/or multiple lines... I'm thinking specifically about the Description and Episode/Show Title.

The web ui provides editor dialogs for certain types of fields. The Description has a "..." button beside it, which pops up a multi-line text box for entry. I could add a "..." to the show title as well, or simply make the default text box length a little larger.

What does "Show Title" display?

Show title is a little bit of a misnomer... it's the MediaTitle field from the custom metadata fields. I think it would help, if I put a help on each field, explaining where/what each field does and where it's stored.

Is the browse feature supposed to display the info in the sagedb or just what is in the properties file? On my setup, sage recordings that have not been run through BMT (They do have the default sage generated EPG metadata) it seems to be displaying the description and not much else. It would be awesome if the browse showed all of the metadata for a given item, similar to how nielms webserver does.... then BMT can be the main interface to view and edit ALL of the metadata (sage generated, bmt generated, default and extra metadata fields)... See the attached jpgs for the differences between what bmt and the webserver shows me. It's entirely possible i missed the boat here... if so just let me know ...[/LIST]

Actually the web ui doesn't use the properties files as all. It will sage to the properties files, but it actually builds the metadata that you see from the Sage Show/Airing/MediaFile objects and the Custom Metadata fields. Currently the code that reads that metadata doesn't read metadata from the show object that would exist in the custom metadata fields. For that reason, in your case, the Show Title (MediaTitle) and Episode Title (EpisodeTitle) and Media Type (MediaType) fields are blank, because they were never scanned and updated. I've added a bug and I'll fix the web ui to pull that extra metadata from the show object if it exists.