Utility: Batch Metadata Importer 3.x

[jaminben]

Originally Posted by stuckless

I had noticed some titles like that as well showing up recently in my collection. I have to check to see if this is a bug in the bmt code, or if the titles actually have quotes in them on imdb or themoviedb.org. I've added a bug for this.

Hiya Sean, Hope you had a good holiday

What I've found is that editing the movie via the Web UI makes no difference. You can remove the associated .properties file, edit the title to remove the "" and as soon as you hit save it will put back the "". There is one way to avoid this and thats to turn off automatic updating but as soon as you enable it again it will put the "" back in.

Cheers

Ben

[stuckless]

Originally Posted by jaminben

Hiya Sean, Hope you had a good holiday

What I've found is that editing the movie via the Web UI makes no difference. You can remove the associated .properties file, edit the title to remove the "" and as soon as you hit save it will put back the "". There is one way to avoid this and thats to turn off automatic updating but as soon as you enable it again it will put the "" back in.

Cheers

Ben

That's interesting (not in a good way ). Yeah.. i'm back... Disney was great.. but i am tired I'm making my way through the bmt/phoenix posts, and i'll try to look at some of these issues today, if i get a chance.

[stuckless]

Originally Posted by wayner

Exactly! My pages in the Metadata Tools web page gets cut off as it is wider than the page but I don't ahve horizontal scroll bars, which I should. On the home page my page is cut off just after status. Or is there some way to adjus the width of the three columns - Phoenix, Metadata Tools and SageTV?

I haven't noticed this issue myself... but I have 24" monitor running at 1900x1600... so I have lots of real-estate to work with I actually turn off horizontal scrollbars in the application, because it messes up the vertical scrollbars... ie, puts them in when they shouldn't be there, etc. What screen resolution so you work in... I can try some tests at that resolution and see what I can come up with.

I might be a little confused about something. Do I need to import the STVi for BMT 3.0? Or can I just use the Web UI? I have just the Web UI going right now and it seems to be working, however, there is a status that is in red that says the automatic plugin on the server is false. Does it need to be imported to run automatically? I tried importing it once, but I seemed to have lost functionality of the web UI. Every time I tried to go to http://Computername : port/bmt/ it would only be some bmt directory that I couldn't seem to do anything with. It wouldn't bring me to the usual bmt status page.

I think razrsharpe mentioned this as well.. but he stvi hasn't changed in a very long time, so no need to re-import it. Also, the stvi and the web ui are in no way connected. ie, importing the stvi, simply updates your STV with the bmt on-demand lookups, and should not impact the web ui. Sometimes if you access the web ui before the jetty service has finished extracting the webapp, you may get the results that you experienced. Also, you can try removing the bmt directory in the webapps directory and restart the server.

Originally Posted by mkanet

Halleluiah! AUTOMATIC PLUGIN SUCCESS!!!!!!!!!!!!!!!!!!!!!!!!

I am happy it finally worked for you. Nothing in the 3.0.2 release changed in terms of where it looked for configuration, etc, but setting the central fanart directory via the web ui, does set the central fanart directory in the server's context. And there is only 1 configuration element for the central fanart folder.

[jaminben]

Originally Posted by stuckless

That's interesting (not in a good way ). Yeah.. i'm back... Disney was great.. but i am tired I'm making my way through the bmt/phoenix posts, and i'll try to look at some of these issues today, if i get a chance.

Not sure if this helps or not, but the "" also appears in the WebUI's search metadata results list, but only for some of the results. If I click on a result which doesn't have the "" then it will still put the "" back once its been saved. It appears that its getting the "" from elsewhere as its not on the WebUI screen until I click the save button.

Cheers

Ben

[wayner]

Sorry if this was covered earlier in the thread but is there any usage documentation for BMT? The documentation link just seems to be for the install and how to get to the web page?

Does BMT automtically attempt to get Metadata for any new/altered media files? I had a show called "Elmo's World - The Great Outdoors" that I renamed to "The Great Outdoors" (since it is in the /Elmo/ folder) and now I have metadata and fanart for the John Candy movie of the same name which automagically appeared. Did BMT fetch the metadata/fanart?

What exactly happens when I do a scan? Does it go through all files selected and put in metadata and fanart, if selected? When you do a scan will it check to see if there is any new fanart for files that have been scanned in the past? How long will a scan take if you have 100s of files in both Videos and TV?

[stuckless]

Originally Posted by wayner

Sorry if this was covered earlier in the thread but is there any usage documentation for BMT? The documentation link just seems to be for the install and how to get to the web page?

I haven't gotten around to creating a usage guide yet... but you can still reference the 2.x reference guide... ignore the installation part.

Does BMT automtically attempt to get Metadata for any new/altered media files? I had a show called "Elmo's World - The Great Outdoors" that I renamed to "The Great Outdoors" (since it is in the /Elmo/ folder) and now I have metadata and fanart for the John Candy movie of the same name which automagically appeared. Did BMT fetch the metadata/fanart?

Yes, if you have the automatic plugin installed. And it uses a fuzzy logic to do the matching, so there's no guarantee that the match will be correct.


What exactly happens when I do a scan? Does it go through all files selected and put in metadata and fanart, if selected? When you do a scan will it check to see if there is any new fanart for files that have been scanned in the past? How long will a scan take if you have 100s of files in both Videos and TV?[/QUOTE]

When you start a scan from the UI, you can "fine tune" what you want to happen. ie, only scan for missing metadata, missing fanart, only update metadata, only update fanart, etc. When you scan from using automatic plugin, it will do a full scan and update metadata and fanart. If you are updating fanart and metadata, a scan for a single file can take anywhere from 10-45 seconds, depending on how many pieces of fanart are being downloaded.

[MitchSchaft]

I have some shows/movies that point to the wrong fanart folder. Is there a way to manually point a video file/tv show to a different fanart folder?

For instance, Men in Black I and II are on TV tonight. I right clicked on Men in Black from the guide, went to Metadata/fanart and selected Men in Black (1997). The art downloaded and everything showed up fine for that movie.

Then I went to Men in Black II in the guide, downloaded the fanart, etc... But, now Men in Black is showing MIB2's fanart . The movie description is still MIB 1, but not the fanart.

Other examples would be Friday the 13th movies. I have all of them and they like to pick and choose which Friday the 13th fanart folder to show from.

[rwc]

Ok, I know this should be obvious, but maybe I"m just slow...so be gentle! :P

I have home movies stored in h:\pictures .. (about 1000 or so) along with thousands of pictures.

Since all the items in h:\pictures is of my family, I know there is know 'fan art' for it.

How do I ignore everything in h:\pictures?!? (I tried adding h:\pictures in the web page for bmt, but it didn't work)

Thanks!

rwc

[Ikarius]

A quick summary of where I'm at with BMT 3.0.2-

I got the metadata to show up on all my media with the method razrsharpe posted- removing directories from my media import directories, rescanning, then re-adding the directories and re-scanning again.

I figured out that almost all the metadata fields I was missing correspond to pre-existing Sage properties, so after the process above, just about all the metadata is there.

What definitely did not work for me was using the "force_full_content_reindex" and/or the sage default STV to kick off a full re-index.

Doing a full content re-index would have one of two outcomes- if I did not have the "missing media items workaround" checked in the web UI, the re-index would work, until it came to the first DVD, that DVD appeared to successfully import the metadata, then the library import thread would crash with an exception.

If the "missing media items workaround" was checked, the library import would go through my entire collection, but all the sage-native metadata fields which were previously empty remained empty. (All the people in roles, year, MPAA rating, etc).

Lastly- BMT appears to be missing something I regard as important- writers! I would assume these would come in as people in a role, like the director, but the writer(s) do not appear to be in the information BMT scrapes, or at least it's not written into any of the properties files. Could this please be added?

Lastly, a couple of web UI nits-
First- checking "do nothing, only scan" in order to browse and manually touch-up the media files is pretty unintuitive. Any chance this could be moved to a new top-level menu item, like "browse"?
Second- when browsing the media files, exactly which version of the metadata are we seeing? Sage's idea of the metadata, or the metadata available in the .properties files? It should be explicit as far as which version of the metadata it is displaying, and ideally, allow you to see either or both.


Cheers
Ikarius

[jsin]

I run only the command line version of BMT; one of the options I use is "--setProperty=bmt/sageMetadata/rewriteTitle:true". Recently I noticed that this option is being ignored with versions 3.02, 3.01, and 3.00. That option worked OK with version 3 beta 9.

No biggie; thought I'd mention it.

[stuckless]

Originally Posted by MitchSchaft

I have some shows/movies that point to the wrong fanart folder. Is there a way to manually point a video file/tv show to a different fanart folder?

The Fanart Folder is derived from the MediaTitle field (i think it's called show title in the web ui). You should be able to set the fanart folder by changing this field.

[stuckless]

Originally Posted by rwc

How do I ignore everything in h:\pictures?!? (I tried adding h:\pictures in the web page for bmt, but it didn't work)

If you are using the exclude filter, then less is better. Try setting the field to just pictures. I think I may have mentioned something about this in the release notes (i think). The exclude filter will exclude any items that "contains" whatever you put in the field. so if you put in "pictures", it will exclude "c:\pictures" or "c:\movies\covers\pictures", etc.

[MitchSchaft]

Awesome, thanks, I'll try that out.

[stuckless]

Originally Posted by jsin

I run only the command line version of BMT; one of the options I use is "--setProperty=bmt/sageMetadata/rewriteTitle:true". Recently I noticed that this option is being ignored with versions 3.02, 3.01, and 3.00. That option worked OK with version 3 beta 9.

No biggie; thought I'd mention it.

I added a bug for it.

[stuckless]

Originally Posted by Ikarius

What definitely did not work for me was using the "force_full_content_reindex" and/or the sage default STV to kick off a full re-index.

Doing a full content re-index would have one of two outcomes- if I did not have the "missing media items workaround" checked in the web UI, the re-index would work, until it came to the first DVD, that DVD appeared to successfully import the metadata, then the library import thread would crash with an exception.

I'm currently trying to track down the source of this issue. I am trying to determine if this is a bmt issue, or a sage issue. The fact the LibraryImporter thread exits with an error leads me to believe it happens outside of bmt, but I want to provide a test case to the sagetv folks before I actually file a bug report.

If the "missing media items workaround" was checked, the library import would go through my entire collection, but all the sage-native metadata fields which were previously empty remained empty. (All the people in roles, year, MPAA rating, etc).

This work around was added becuase it appears that if I return the properties, then sagetv errors out, but if I return null (which is the work around), then sage carries on. In some cases, sage will then re-read .properties, but in other cases it does not. I think the best approach is for me to understand exactly why it fails in some cases for ONLY dvds. This is a time consuming process, because I think it may be data related, but I not certain.

Lastly- BMT appears to be missing something I regard as important- writers! I would assume these would come in as people in a role, like the director, but the writer(s) do not appear to be in the information BMT scrapes, or at least it's not written into any of the properties files. Could this please be added?

I get writers in my properties files and in the metadata that shows up in sage. Perhaps you could give me an example... tv show, movie, and which provider (or just send/post a .properties file since it will have that information as well).

Lastly, a couple of web UI nits-
First- checking "do nothing, only scan" in order to browse and manually touch-up the media files is pretty unintuitive. Any chance this could be moved to a new top-level menu item, like "browse"?

Yes. Eventually there will be a separate browser, but for now, I added in this option for the purpose of doing a manual scan. Maybe I'll see about adding a simply menu entry for browing that just jumps to the manual scan screen.

Second- when browsing the media files, exactly which version of the metadata are we seeing? Sage's idea of the metadata, or the metadata available in the .properties files? It should be explicit as far as which version of the metadata it is displaying, and ideally, allow you to see either or both.

Good suggestion. You are currently seeing what sage sees, which is why sometimes when you "update" an item, you appear to lose your changes. The changes are written to the .properties, but because of timing, when the UI re-reads the metadata, it does so from the sage object and not from the .properties. It would be fairly easy to allow a user to switch between .properties and what sage sees. I'll add an enhancement request for this.

[MitchSchaft]

Alright, that did the trick for me. Manually changing the name in the UI. I'm gonna have to turn off auto metadata/fanart or else it'll go back to the other movie. Ben from 1972 is an offender. It keeps pulling data for Ben X from a couple years ago. I should start contributing stuff to themoviedb.org. Maybe that will solve the problem.

[can3gxw]

After some searching, I am unable to find an answer to this question... if there is an answer already, please let me know...

Is there a way to tell BMT to IGNORE a specific folder within the regular Sage Imported Media directories so that it doesn't look for fanart for ALL the media?

One example: I have the folder "VIDEOS" set up in Sage as an "Imported Media Folder", and when new files are dropped there (movies, tv shows, etc), BMT will go out and find all the fanart and metadata for each file in that folder, as well as any subfolders. HOWEVER: I also have a subfolder there called "MISC" where I throw anything else that I just want to watch/see in Sage, but don't want BMT to do anything with it.

Yesterday, I imported a DVD rip from my son's grade 4 class project and just called it "School". BMT found it, and proceeded to download the fanart and metadata for some documentary called "School".

Is there a setting somewhere that I can tell BMT to IGNORE anything in the subfolder "MISC" and don't download any metadata for that subfolder?

[jaminben]

Originally Posted by can3gxw

Is there a way to tell BMT to IGNORE a specific folder within the regular Sage Imported Media directories so that it doesn't look for fanart for ALL the media?

Try looking in Metadata Tools > Media Configuration > Ignore These Folders, maybe this will do what you want.

[can3gxw]

LMAO! Geez.... couldn't be much easier, could it??? If it were a snake, it'd have bitten me!

Thanks! I guess I need to go through each setting and see what other goodies I'm missing.

[can3gxw]

Well... now it seems like I've blown "everything" up...

I added the path (relative to the server) "d:\videos\misc" to the option "folders to ignore", since this is the one that I want to ignore in the BMT updating... now all the "Failed Media Files" are showing "NoName" and the process is flying through my media... not sure what it's doing...

Can anyone confirm what the path syntax should be in that dialog box?