Utility: Batch Metadata Importer 3.x

[jonnydeath]

Originally Posted by stuckless

Jonny, there isn't much in the log files But i'm wondering if this is related the phoenix version. I noticed that you are using the latest 1.58 (which is basically a developer version). Just try extracting the 3.1 bmt again, which is overwrite the phoenix.jar to an older version, and then restart sagetv server and see if that fixes it.

I went back to 3.1 and it did not work. i went to 3.0.2 (only other one i had) and it worked. I updated to phoenix 58 and it still worked. i don't know why my system doesn't like 3.1.

[superdodge]

stuckless,

Could this be related to my problem with the automatic plugin not running on new recordings? It's taken from mailcheck.log

2010-01-27 19:45:51,263 ERROR [AWT-EventQueue-0] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files (x86)\SageTV\SageTV\.\STVs\Phoenix\Configuration\Phoenix.xml

[stuckless]

Originally Posted by jonnydeath

I went back to 3.1 and it did not work. i went to 3.0.2 (only other one i had) and it worked. I updated to phoenix 58 and it still worked. i don't know why my system doesn't like 3.1.

I don't know how to explain that. It's too bad that 3.1 gave you so much issues.

Originally Posted by superdodge

Could this be related to my problem with the automatic plugin not running on new recordings? It's taken from mailcheck.log

Actually that "error" message is harmless (for now)... and I've been trying to change much of the code log "warnings" instead of errors in cases like these.

[alfi33]

Originally Posted by appelm

Was the above ever resolved? I'm sorry if this has been covered somewhere.. I couldn't find it and am currently having a problem with too many genres. I've occasionaly had to replace a drive and my library gets re-imported. Then I have multiple genres ('action' & 'action film', etc..). It actually freezes my extenders when I page down in SageMC.. I'm not sure if its a SageMC issue or an extender issue.. but if I simply go through all my movies and remove the 'xxx film' categories thereby reducing the number of genres, my extender doesn't freeze. An automated solution like the above idea would be great.. Thanks!

+1
Just having the ability to add, edit, or remove Genres from your web gui would be very handy...even if the fancy genres lookup table didn't make it in there right away. Thanks for all your work on this, stuckless!

[stuckless]

Originally Posted by alfi33

+1
Just having the ability to add, edit, or remove Genres from your web gui would be very handy...even if the fancy genres lookup table didn't make it in there right away. Thanks for all your work on this, stuckless!

The genre mapper will be there for 4.0 (since I don't think it will be that hard). I never really considered doing a web ui to manage mappings... but I'll add it as an enhancement request.

[Oats]

I seem to be having an issue getting metadata for The X-Files series. For some reason when I scan using the web interface it isn't finding anything. I used the normal naming that I use for all my files, ShowName SXXEXX - EpisodeName.

If I manually search it finds the metadata fine. Also it is working fine on other shows, just not any of the X-Files rips.

Any idea what could be going on? I don't really want to manually search for the data since there are over 200 episodes.

[stuckless]

I think this is a "flaw" in 3.x. Basically when 3.x does a title search it stips non alpha-numeric characters, so the search becomes "x files" and not "x-files". This causes an issue in tvdb, since it's matching requires the '-'. I have fixed this in 4.x (not released).

[Oats]

That it explains it, thanks. Now to decide if I want to go through them all or wait until 4.x comes out.

[stuckless]

tvdb really doesn't have a good engine. For example, "X Files" returns nothing, while "XFiles" does

If your files were named "The XFiles S1E1.av", then bmt could find it, but "The X-Files S1E1.avi" returns nothing from tvdb.

BTM 4, is hopefully going to go beta in the next couple of weeks, and depending on the types of issues uncovered, the full release won't happen for another 4-6 weeks after that.


Just an update for those interested in the 4.x release...
* As I just mentioned, I'm hopefully going to release a beta in the next couple of weeks. I'm currently running it on my server, and I've uncovered some issues.
* The web ui get getting some major updates. Rather than the simply "scan" model introduced 3.x, you'll be able to browser and navigate your media collection, and the run a "scan" from a particular location. ie, if you are in the "Recordings" view, you can scan from there, and it will only scan the "Recodings".
* There are more options. ie, the when you scan, you will be able to prevent bmt from actually updating the wiz.bin directly. You can also optionally turn off the ability to create .properties files, if you are updaitng the wiz.bin, etc.
* The metadata-titles has been replaced with a more flexible look up system. the metadata titles was only ever consulted if a search happened, and it returned results... you could specify which result to choose in the metadata titles. This works ok in some cases, but fails big time, when you consider the problem about with "X-Files". So, now a new file is used, and it will be consulted BEFORE the search happens. You can do things like directly force the tvdbid for all files in the given subfolder, etc.
* Searching will now use the year when selecting results. ie, if your filename has a year it in, then when selecting a result it will select the result based on title match and year. (currently titles only matched on title only)

There are lots of other smaller enhancements, such as better metadata editing from the web ui, more commandline options, automatic plugin will use the message system to related failed scans, etc...

I'll provide more details later, but it's getting close to beta time

[SteveW]

I've run into a little issue. Searches on the error turns up some matches on the errors, but not the situation...

I installed BMT on my Sage server. Ran up the UI through Jetty, looked great. Ran a manual scan and setup the Fanart.

Decided I wanted to load the automatic plugin in SageMC, so did that, followed the instructions that said the sage server would need to be restarted, so did so.

Now the UI doesn't work anymore. I get "Failed to get status" on SageTV, Phoenix, and Metadata Tools when I run the web browser on the server.

If I try to run the UI from my other PCs, I just get a directory listing in the browser of /BMT/ and what appears to be the contents of the .WAR file.

Any suggestions on straightening this out?

Tnx

[razrsharpe]

@SteveW: could be that the context xml file that ships with 3.1 does not require authentication... before i upgraded the context xml file to one that requires authentication i had the same problem. Get the updated xml here (the previous post has the instructions on what do with it).

[SteveW]

Originally Posted by razrsharpe

@SteveW: could be that the context xml file that ships with 3.1 does not require authentication... before i upgraded the context xml file to one that requires authentication i had the same problem. Get the updated xml here (the previous post has the instructions on what do with it).

OK, tried that. I now have to login before I get to the web UI, but I get the same errors again as I listed in my previous post after login.

[stuckless]

Steve,

Try
* shutting down sage tv
* remove the jetty/webapps/bmt/ directory (not the war)
* start sagetv
* wait a few seconds
* restart your browser
* connect to the the bmt url

See if this works...

[SteveW]

Originally Posted by stuckless

Steve,

Try
* shutting down sage tv
* remove the jetty/webapps/bmt/ directory (not the war)
* start sagetv
* wait a few seconds
* restart your browser
* connect to the the bmt url

See if this works...

Well that's pretty funny. I had read something somewhere else about removing that directory, and was trying that at the exact same time you were posting this!

Happy to report that fixed my problem!

Tnx!
-S

[SteveW]

Originally Posted by SteveW

Well that's pretty funny. I had read something somewhere else about removing that directory, and was trying that at the exact same time you were posting this!

Happy to report that fixed my problem!

Tnx!
-S

Hmmm. I just had this happen again after restarting my Sage server. Is it necessary every time the server is restarted to delete the /bmt/ directory?

-S

[stuckless]

Originally Posted by SteveW

Hmmm. I just had this happen again after restarting my Sage server. Is it necessary every time the server is restarted to delete the /bmt/ directory?

-S

That can be fixed with the information from this post. The context xml that razrsharpe and I referenced in a previous post was an older one. This is a newer one, that should fix the problem. I've also updated the first post to include a reference to this fix as well.

So, shutdown sage, remove the web dir, and then update the context xml file, with the one provided, and start sage. It should be good after that.

[bikesquid]

Been playing around with this for a bit and it looks like a great tool. I've got a question about importing existing data.

With 5000+ episodes saved I'm not going to be able to do this one show at a time....

Is it possible to import using existing metadata ? coming from vista media center I've been using metabrowser to gather metadata for all my files. My structure looks like this:

G:\Alias\series.xml
G:\Alias\Season 1\S01E01.mpg
G:\Alias\Season 1\folder.jpg
G:\Alias\Season 1\backdrop.jpg
G:\Alias\Season 1\metadata\
G:\Alias\Season 1\metadata\S01E01.xml
G:\Alias\Season 1\metadata\180267.jpg (image file for episode... referenced in s01e10.xml)

I'd like to just import this existing info into sage, specifically I'm looking at TV Explorer to integrate what I've got with the new stuff being recorded by sage.



Any suggestions?

[stuckless]

There is nothing out of the box that works with this layout in bmt. The best that could be done, would be to have a "metadata provider" that could convert your data into something that sage/sagemc/tv explorer could consume. My time is pretty scarace these days, but if you zip up a few .xml files files and send them to metadatatools at gmail dot com, then I can take a look at them sometime and see what I can do.

[bikesquid]

Originally Posted by stuckless

There is nothing out of the box that works with this layout in bmt. The best that could be done, would be to have a "metadata provider" that could convert your data into something that sage/sagemc/tv explorer could consume. My time is pretty scarace these days, but if you zip up a few .xml files files and send them to metadatatools at gmail dot com, then I can take a look at them sometime and see what I can do.

Wow, if you were able to sort a solution that would really save my bacon!
a few sample file sent. More can be sent if that would help.

if there's anything I can do to assist, my time is yours.

I'm still trying to get bmt to work with sage recordings as well, so maybe I'm having a more serious configuration issue going on.

[stuckless]

Originally Posted by bikesquid

I'm still trying to get bmt to work with sage recordings as well, so maybe I'm having a more serious configuration issue going on.

The first thing that I'd do, is see if those shows are actually defined in tvdb. If not, then that will account for the Failed. If they are, then email the log file (sagetv_0.txt) to me, and I do a quick check to see if there is another underlying issue.