Utility: Batch Metadata Importer 3.x

[bialio]

Originally Posted by stuckless

The "default" fanart dir is the fanart dir that would be used, if there were any fanart. ie, it's not an indicator that there is fanart, just shows you where you would need to put fanart for it to be picked up.

Is this a show that you scanned using the web ui?

for TV fanart, it does try to check the season folders folders first, and then fall back to the non season folders if there isn't season specific fanart. (although i did recently find a bug in the season code recently)

I'm pretty sure it got into this state after I did a Scan from web for Recordings without fanart, and told it to only update the fanart.

[bialio]

OK - this turns out to be a combination of the weird linking of airings in BMT and the way the STVi that I'm working on pulls fanart - I'm basically getting the first episode in an array of episodes and using it to get the fanart.

The first episode in the array is the newest, and it had bad metadata after the web gui scan.

[stuckless]

Originally Posted by bialio

I'm pretty sure it got into this state after I did a Scan from web for Recordings without fanart, and told it to only update the fanart.

Originally Posted by bialio

OK - this turns out to be a combination of the weird linking of airings in BMT and the way the STVi that I'm working on pulls fanart - I'm basically getting the first episode in an array of episodes and using it to get the fanart.

The first episode in the array is the newest, and it had bad metadata after the web gui scan.

Ok... here's my approach. I'm going to finish the 4.0 release, but in the 4.0 release, i'll provide a better approach to updating the wiz.bin. Either, i'll fix what I have by comparing (once again) to nilems import, or I'll provide a mechanism to NOT update the wiz.bin but still allow the fanart metadata (ie, custom metadata fields) to be updated, so that sage Recordings will never lose their metadata.

The first couple of releases of 4.0 will be beta... so if you are brave enough, in a couple weeks, you can try the betas to see if the problem still exists in the betas.

[bialio]

I'm brave! It's just metadata right?

Although a nice feature would be the ability to 'clean' the wiz.bin of everything meta-related - mainly it's the viewing history that needs to be preserved. Then if the metadata gets completely hosed we could at least clear it all and start 'clean'

Of course, it occurs to me that I have backups of wiz.bin every night on my WHS machine, so I guess I could figure out when I installed BMT and go back to that wiz.bin.......

btl.

[Skybolt]

Originally Posted by stuckless

...Bottom line, I'm not sure that Shared DVD drive fanart is possilble.

I agree, but where is the metadata stored?
Is it in the wiz.bin, because after I updated from IMDB I got new cover art and description. I did this thinking I could edit the metadata, cover art latter.

Thanks,
Skybolt

[Tiki]

I experienced the "weird linking problem" for the first time today. I thought I would post my details in case it helps Stuckless with debugging...

Last night, I used the plugin to manually search for fan art / metadata for an episode of "Castle" that I had just finished watching. The plugin found multiple matches - I chose one called "Castle (2009)". The fan art downloaded and all appeared OK. I then deleted that episode of Castle since I had finished watching it (I just wanted to have the fanart for future recordings).

I didn't notice a problem until today. When I went to watch a recorded episode of "Sanctuary", I saw that it was displaying the fanart for Castle. Sanctuary had been recording last night around the same time that I downloaded the fan art for Castle. This may have something to do with how they got "cross-linked". Aside from the fan art being wrong, all of the show details displayed inside Sage for this episode of Sanctuary appeared correct. Also, if I did a search for Past Airings of Sanctuary, all other episodes had the correct fan art (even other airings of the same episode) - it was only this particular airing that was corrupted.

I checked the metadata-titles.properties file in the SageTV directory. I found both Sanctuary and Castle (2009) listed withe the correct TVDB id codes - they were not swapped.

I checked in the recording directory for the actual MPG file for Sanctuary to see if there was a properties file with the wrong info - there was no properties file.

I used the "Browse" function in the BMI Web UI to find the corrupted Sanctuary and found that all of the data was correct, except "Show Title", "Episode Title", and "Metadata ID" (and of course the fanart information).

As a test, I corrected these 3 fields and hit the "Save" button in the BMI Web UI. After that, Sage displays the correct fan art (but loses the channel number and channel name values, so the channel logo no longer appears in the recordings menu). Also, a properties file is created in the recording directory. So, it appears that properies files get created when you save data from the Web UI, but not when you import data by doing a manual search with the STVi plugin. I don't have an opinion on whether this is good or bad at this point - this is just an observation.

This did expose one other problem. When I downloaded the data for Castle through the plugin, it renamed the show title as "Castle (2009)" to match the TVDB website database. It also stored all the fan art in a folder called "Castle (2009)". The problem with this is that future episodes of Castle recorded through Sage (or looked at in the TV Guide, Past Airings, Future Airings, or Watched History) will not be able to find that fan art since the title is different. For TV shows, it would be better if it left the show title alone and named the fan art folder to match the title of the recording.

Thanks again for all your hard work Stuckless - this really is an amazing tool!

My setup details when this incident occurred:
SageTV 6.6.2
Metadata Tools 3.1
Phoenix API 1.44
Sagex API 6.6.1
Java 1.6.0_17
Default STV with BMITools_forDefaultSTV.stvi v1.0
Windows 7-64 bit

[bialio]

SO - after playin' with BMT for about a month now my wishlist for 4.0:

1) No weird linkage of airings
2) Option to update only empty meta-data fields (ie no losing of the default Sage EPG data)
3) I think this was mentioned up above, but when the title of the show doesn't quite match a scrapers hit - like the "Castle (2009)" example above, that causes complications. For me it's 'Curious George (2006)' - all the recordings I have archived of Curious George are suddenly showing up with the date, which is annoying. If #2 makes it in then this might not be such an issue.

btl.

[stuckless]

Originally Posted by bialio

1) No weird linkage of airings

That's definately on the list for 4.0

2) Option to update only empty meta-data fields (ie no losing of the default Sage EPG data)

That's on the list for 4.0 (I'll add an option to ONLY update non-null metadata)

3) I think this was mentioned up above, but when the title of the show doesn't quite match a scrapers hit - like the "Castle (2009)" example above, that causes complications. For me it's 'Curious George (2006)' - all the recordings I have archived of Curious George are suddenly showing up with the date, which is annoying. If #2 makes it in then this might not be such an issue.

I'm not quite sure I fully understand. Is this a "false positive" type of scenario, where it thinks curious george is some other show? If so, then there are a couple of new additions to 4.0 that might help.
1. All searches will now take the year into account, so if you a year in your title, then you'll get better hits (especially in cases where there are multiple movies/shows of the same name for different years)
2. The media-titles.properties has been removed and replaced with an xml file that allows you to set some defaults for a query based on regular expresssions of file matching. It's for advanced used, but it would allow you to do stuff like explicitly set the series id for a show if it's in a particular folder, etc.

Btw.... everyone should look over the outstanding bugs, and add any new issues that you'd like to see fixed in 4.x. In terms of bugs/enhancements, I'm only working off this list. Issues that have a "release 4.0" means that they have been already fixed in 4.0.

[stuckless]

Originally Posted by Tiki

I checked in the recording directory for the actual MPG file for Sanctuary to see if there was a properties file with the wrong info - there was no properties file.

The stvi plugin does not create a .properties... it just updates the sage metadata directly. I'll add an issue to look at this, so that it's consistent.

I used the "Browse" function in the BMI Web UI to find the corrupted Sanctuary and found that all of the data was correct, except "Show Title", "Episode Title", and "Metadata ID" (and of course the fanart information).

As a test, I corrected these 3 fields and hit the "Save" button in the BMI Web UI.

This is interesting, because it's different than bialio's issue. He let me use the web ui on his home collection, and when I would update the metadata for 1 item it would update it for the "linked" item as well. In his case, they were truly "linked", but in your case, it looks like only the custom metadata fields had the wrong value, but your files were not actually "linked". (And I have no idea how 2 files becomes linked, since from what I can tell from bialio's files, they had different media ids, airing ids, show ids, etc.)

After that, Sage displays the correct fan art (but loses the channel number and channel name values, so the channel logo no longer appears in the recordings menu).

I've added a bug for this. Hopefully it will get cleaned up when I figure out what to do with the Show/Airing updating code.

This did expose one other problem. When I downloaded the data for Castle through the plugin, it renamed the show title as "Castle (2009)" to match the TVDB website database. It also stored all the fan art in a folder called "Castle (2009)". The problem with this is that future episodes of Castle recorded through Sage (or looked at in the TV Guide, Past Airings, Future Airings, or Watched History) will not be able to find that fan art since the title is different. For TV shows, it would be better if it left the show title alone and named the fan art folder to match the title of the recording.

I've added bug for this as well. I'll try to parse out the Title if it contains a (YEAR). I do this now for IMDB because it always returns a Title (YEAR) string.

[jryan15]

hello all, i'm having some trouble with the automation. a scan was completed and all of my metadata was pulled in, but none of the fan art did. when i use the web interface to look up metadata, it does grab fan art and it shows up in my moviewall. i'm just not sure how to configure bmi to do this automatically...

i should note that i'm running windows 7 and sage in service mode. when the web interface pulls in meta data, it goes to the phoenix directory within c:\progam files, but when i manually use the "import fanart\metadata" button in the moviewall options, the fanart goes to the phoenix directory in c:\user\htpc\appdata\virtualstore\... directory.

any help would be greatly appreciated!

thanks.

[stuckless]

Originally Posted by jryan15

hello all, i'm having some trouble with the automation. a scan was completed and all of my metadata was pulled in, but none of the fan art did. when i use the web interface to look up metadata, it does grab fan art and it shows up in my moviewall. i'm just not sure how to configure bmi to do this automatically...

Seems odd that no fanart is being downloaded at all. Maybe you could sent me your logs to metadatatools at gmail dot com.

i should note that i'm running windows 7 and sage in service mode. when the web interface pulls in meta data, it goes to the phoenix directory within c:\progam files, but when i manually use the "import fanart\metadata" button in the moviewall options, the fanart goes to the phoenix directory in c:\user\htpc\appdata\virtualstore\... directory.

This sounds like a class windows UAC issue. I'm assuming that you are running a client and not a placeshifter or extender. If you are running a client, and it is NOT running as the same user as the sagetv server, then there is no way that the client is going to populate data in the sagetv home dir.

probably the best thing to do, is to set the fanart directory to someplace outside the sagetv home directory, ie, c:\fanart\. Then from the webui, configure the fanart dir, and then in the client, make sure you configure the fanart dir as well. ie, both the client and the server must configure their fanart locations. I would also delete the fanart from the virtual store area, since it should never be there.

I'm not a very well versed in the obfuscated windows security techniques, but I think there is a UAC sticky somewhere that talks about this, and there are several extremely smart windows folks here, that may be able shed more light on what needs to be done exactly.

FYI... If this is a UAC issue, then you may also encounter some issues when you go to upgrade certain plugins (ie, bmt) as well. If you do not update your plugins using the same user as the sagetv service user, then I think that windows will quietly write those updates to the virtualstore (and never inform you), and they will never be seen by the real sagetv service user. (i think)

good luck... and feel free to email your logs,if you don't figure this out.

[bialio]

One other feature request - it'd be nice if there was a way to only save the fanart (or conversly only the metadata) from web based 'Find Metadata' on a particular recording / video / DVD - ie the same option that there is in the automated scan.

btl.

[stuckless]

Originally Posted by bialio

One other feature request - it'd be nice if there was a way to only save the fanart (or conversly only the metadata) from web based 'Find Metadata' on a particular recording / video / DVD - ie the same option that there is in the automated scan.

btl.

unfortunately, i don't have 3.1 up and running right now... but doesn't the save... button bring up a dialog with options, "overwrite fanart", "overwrite metadata", etc?

ie, when you do find metadata, it may bring back the updated metadata, but when save it, if you uncheck overwrite metadata, then it will not persist that metadata, but it should still download the fanart... (i think).

Or, are you asking that when you do a find metadata, that you have the option there, so that it won't even bring back updated metadata, if overwrite metadata is not checked. I can certainly add that to the lisa as well.

[bialio]

When I hit save I just get a grey toast saying that it's working on saving it, then a green toast with "Saved."

btl.

[ohpleaseno]

For some reason the rename mask for Episodes in Recorded TV keeps changing. I have my recordings grouped by show, so when I select "How I Met Your Mother", it will list all recent recordings as "How I Met Your Mother" and not the episode title "Jenkins". If I change it back, no problem, but this is the second time it has happened and I hardly ever even go into the BMI web ui, only if this happens, so I haven't been changing any settings.

[jryan15]

Originally Posted by stuckless

Seems odd that no fanart is being downloaded at all. Maybe you could sent me your logs to metadatatools at gmail dot com.

Looks like I've got fanart updating now via the web interface, and all is working ok now. The latest reinstall must have took care of the issue (probably operator error on the previous attempts ). Thanks.

[jryan15]

I was wondering if someone could point out the acceptable file naming method for avi's that have bene split into multiple files ie cd1, cd2. I know that i read a post on it a few days ago, but I can't find it.

Also what is the appropriate method for renaming and moving a folder and avi file after it has already been processed by bmi? Is it ok to manually move and rename the file and then delete the associated properties file?

One more question (sorry), I have a few movies that found metadata but no cover art. I manually found good cover art for the movies, can I just copy/paste the art into the appropriate fanart directory? Do I need to manipulate the properties files?

Thanks!

[stuckless]

Originally Posted by ohpleaseno

If I change it back, no problem, but this is the second time it has happened and I hardly ever even go into the BMI web ui, only if this happens, so I haven't been changing any settings.

Did you verify in the web ui, that the title tv epsidode title mask is in fact changing? It is odd, since nothing in bmt, actually sets that property, except for the web ui. Now, if the property is erased from your Sage.properties, then bmt will default it back to the default value.

Originally Posted by jryan15

I was wondering if someone could point out the acceptable file naming method for avi's that have bene split into multiple files ie cd1, cd2. I know that i read a post on it a few days ago, but I can't find it.

I'd use... "Filename (Year) cd1.avi", or omit the year "Filename cd1.avi". If the year is in the filename, then in 4.0 (not released), then it will be used.

Also what is the appropriate method for renaming and moving a folder and avi file after it has already been processed by bmi? Is it ok to manually move and rename the file and then delete the associated properties file?

If the information has been imported into the sagetv database(wiz.bin), then you can remove the properties, but personally i'd keep them, for a backup.

One more question (sorry), I have a few movies that found metadata but no cover art. I manually found good cover art for the movies, can I just copy/paste the art into the appropriate fanart directory? Do I need to manipulate the properties files?

I think the web ui will show you the fanart folder... so if you drop the .jpgs in that folder, then it should work.

[jryan15]

Great! Thanks!

Originally Posted by stuckless

If the information has been imported into the sagetv database(wiz.bin), then you can remove the properties, but personally i'd keep them, for a backup.

I guess my concern was with the line in the properties file that references the file name and location and if a discrepancy would cause problems. Is the properties file used when I browse or scan using the web interface?

[stuckless]

Originally Posted by jryan15

I guess my concern was with the line in the properties file that references the file name and location and if a discrepancy would cause problems. Is the properties file used when I browse or scan using the web interface?

Any filenames in the properties file is for reference only. The webui uses the sagetv metadata from the wiz.bin... it never reads the properties file directly.