Utility: Batch Metadata Importer 3.x

[stuckless]

Originally Posted by jaminben

Does this mean we would always have to manually add metadata from the web UI? This would be disastrous from my point of view.

Originally Posted by babgvant

Please don't do that. The automatic plugin is the best part, and why I euthanized my metadata tool.

Originally Posted by gplasky

NNNNNNNNNNNOOOOOOOOOOOO!!!!!!!!!!!!!
That's what makes this such a joy to use. (I love the WebUI too.)This is what got me to move away from using DVD Profiler. (And I can't use the euthanized metadata tool anymore.)

Ok... Ok... I hear you... I won't remove the plugin... just thought removing it was easier than fixing it

Originally Posted by sflamm

Why not just code your plug-in to be re-entrant? You know, like an interrupt handler... then you wont have an endless loop.

Ultimately that's what needs to happen... just right now the process is stateless and easy to manage. Doing this is just a little more complicated, but it's the only way to really solve the issue.

[jaminben]

Originally Posted by stuckless

Ok... Ok... I hear you... I won't remove the plugin... just thought removing it was easier than fixing it

[sflamm]

Ultimately that's what needs to happen... just right now the process is stateless and easy to manage. Doing this is just a little more complicated, but it's the only way to really solve the issue.

You can do it!

Just add one static variable (semaphore) and check state on entry, and return immediately as necessary (just like resetting an interrupt handler immediately)

Notice how quickly the board lit up in response to the 'automatic plug-in'. That's the golden goose baby. Ideally that's all anyone would use.

[can3gxw]

Originally Posted by stuckless

If the episode is in thetvdb then it should work. If the episode is in thetvdb and it's sitll not working, then let me know, and i'll run some tests.

Hey Sean... I guess lots of work being done this weekend from these posts... being a programmer myself, I feel for ya... people don't know what they have until they need something fixed, then they can't live without it!!

I checked the TVDB today for this episode of Bizarre Foods and it was there - S01E14. When I made a copy of the file and renamed it to Bizarre Foods S01E14 it worked perfectly.

Whenever you can find the time...

[Fuzzy]

I think what might work is just to move away from being implemented as a metadata source. Instead, have the request for metadata from sage simply add the requested file to a queue, and let that queue handle the metadata search like the web-ui or command line do. That way, you can still use sage's getmetadata calls, but not worry about loops, as all it would trigger would be an attempt to re-add the file to the queue (obviously, you'd have to have a check to see if it was already in the queue, and ignore).

[stuckless]

Originally Posted by can3gxw

Hey Sean... I guess lots of work being done this weekend from these posts...

We'll see... I'm starting to get addicted to Lost... so it's eating into my spare time. I've just started Season 2

I checked the TVDB today for this episode of Bizarre Foods and it was there - S01E14. When I made a copy of the file and renamed it to Bizarre Foods S01E14 it worked perfectly.

Here's the problem... Your filename is BizarreFoodsWithAndrewZimmern-BestBites-173758-0.avi when means the BMT will pass the following title to tvdb, "Bizarre Foods With Andrew Zimmern", and it returns no results. tvdb does have an entry for "Bizarre Foods", but not for the full title. Unfortunately for you, there is nothing you can really do until I add a better metadata-titles support

Originally Posted by Fuzzy

I think what might work is just to move away from being implemented as a metadata source. Instead, have the request for metadata from sage simply add the requested file to a queue, and let that queue handle the metadata search like the web-ui or command line do. That way, you can still use sage's getmetadata calls, but not worry about loops, as all it would trigger would be an attempt to re-add the file to the queue (obviously, you'd have to have a check to see if it was already in the queue, and ignore).

I thought about this as well, and actually this is my preferred solution, but it's more work I may start out with sflamm's idea, and then move to this one later.

[Fuzzy]

Well, one bonus advantageof the queue idea is that you can unify pretty much everything into one set of code. The command line app, the web ui, and the import, would all just be different interfaces to the queue management system. At least, that's how I can envision would be most robust. The queue basically maintains a list of files that need updating. It starts at teh top, and works its way down. As it goes, if it encounters a failed grab, it marks it as such, and continues down the list. This way, you could periodically check the web ui and see what is remaining, and make the manual decisions required.

Obviously, a major departure from current, but might be a direction to think about for 3.5 or 4.0... ;-)

[stuckless]

Originally Posted by Fuzzy

Obviously, a major departure from current, but might be a direction to think about for 3.5 or 4.0... ;-)

I agree... But it'll be a 3.5 or 4.0 feature

[nyplayer]

Bmt 3.1,

BMT seemed to have hosed a lot of my recordings that had the episodes. for example Jerico now says Jerico -Jerico instead of the episodes and they all have the same airdate also same description. I removed bmt deleted all the properties but that didn't help. I guess I am stuck.... with the incorrect metadata. Also it did the same with the Twilight zone.

It appears that if there is a movie or anything else with the name of the series it messes up the metadata. I guess it I should have never done a full scan.

Example of a messup. Look at the episode name. (WAF bad after this) file is JonKatePlus8-KoreanDinner-3829706-0.mp4

<?xml version="1.0" encoding="utf-8" standalone="no" ?> 
  <!DOCTYPE sageShowInfo (View Source for full doctype...)> 
- <sageShowInfo version="1.3">
- <channelList>
  <channel channelId="0" /> 
  </channelList>
- <favoriteList>
- <favorite favoriteId="99873">
  <title>Jon & Kate Plus 8</title> 
  <firstRun /> 
  <reRun /> 
  <quality>Default</quality> 
  <stopPadding>60</stopPadding> 
  </favorite>
  </favoriteList>
- <showList>
- <show epgId="EP9102460095">
  <title>Jon & Kate Plus 8</title> 
  <episode>Jon & Kate Plus 8</episode> 
  <duration>1346</duration> 
  <category>Reality</category> 
  <subCategory>Parenting</subCategory> 
  <description>With two year-old sextuplets and six year-old twins, the Gosselins are hardly your typical American family. Jon and Kate are the ambitious parents of this adorable bunch and they are battling all odds to make sure their brood has a normal, happy childhood. We follow them as they tackle seemingly ordinary life events like pumpkin picking and birthday parties that become extraordinary when you have two sets of energetic multiples.</description> 
  <year>2007</year> 
  <originalAirDate>2007-04-16T07:00:00.00Z</originalAirDate> 
- <airing channelId="0" duration="1346" favoriteId="99873" sageDbId="4075283" startTime="2009-09-01T00:00:00.34Z">
  <watched /> 
  <manualRecord /> 
  <recordSchedule duration="1346" startTime="2009-09-01T00:00:00.34Z" /> 
- <mediafile duration="1346" sageDbId="3934336" startTime="2009-09-01T00:00:00.34Z" type="TV">
  <archived /> 
- <segmentList>
  <segment duration="1346" filePath="\\192.168.2.155\j$\sageuncj\JonKatePlus8-KoreanDinner-3829706-0.mp4" startTime="2009-09-01T00:00:00.34Z" /> 
  </segmentList>
  </mediafile>
  </airing>
  </show>
  </showList>
  </sageShowInfo>

[bjwise96]

First, thanks for a great utility.

I'm having trouble with ripped TV DVDs. I've got two scenarios: full disk rips and episode rips. The full disk rips work fine, but I can't get the episode rips to work. Here are two examples:

Burn Notice\Season 2\Burn Notice S02E05.mkv
The Big Bang Theory\Season 1\The Big Bang Theory S01D03

The disk works fine, but the episodes do not. Any idea what I'm doing wrong? I've done tons of reading in this thread and searching, so sorry if I just missed something simple.

I'm using the default settings on BMT 3.0.2. I'm also using Phoenix 1.42 and SageTV 6.6.2.218 and SageMC 6.3.9

Thanks in advance and thanks for a great tool!

Bryan

[stuckless]

Originally Posted by nyplayer

BMT seemed to have hosed a lot of my recordings that had the episodes. for example Jerico now says Jerico -Jerico instead of the episodes and they all have the same airdate also same description. I removed bmt deleted all the properties but that didn't help. I guess I am stuck.... with the incorrect metadata. Also it did the same with the Twilight zone.

It appears that if there is a movie or anything else with the name of the series it messes up the metadata. I guess it I should have never done a full scan.

This is precisely why I was reluctant to add TV support into bmt... I hope you backed up your metadata, because if not, then you hosed. BMT is not a miracle worker, and when it doing fuzzy logic title matching, then there is going to some room for error. The problem in your case is the that bmt selected an exact match from the TVDB for "Jon & Kate Plus 8", but the episode that you wanted was called, "Jon cooks Korean Dinner", not "Korean Dinner", so it didn't do an exact match on the episode. When that happens, bmt still tries to scape some data and bring it back.

Sure, you can argue, that well, bmt should have [insert your personal wish here], but the reality, is that this is one 1 person team, that is stretched pretty thin.

The best thing you can do, to protect yourself again this type hosing, is to make regular backups, and NEVER use the automatic scanner. The web ui has a manual lookup scanner, so you can use that instead of the automatic scanning. Automatic scanning should only be used by people that are OK if they don't 100% correct results (which is me).

From the web ui, you can do manual scans for your jon and kate episodes. For the jon and kate episodes, enter the season and episode #s in the search box, and it will bring down the metadata.

I know how frustrating this is... but bmt cannot, unfortunately, make all the right decisions when it's running as an automated tool.

[stuckless]

Originally Posted by bjwise96

I'm having trouble with ripped TV DVDs. I've got two scenarios: full disk rips and episode rips. I can't get the episode rips to work.

Burn Notice\Season 2\Burn Notice S02E05.mkv

I'm using the default settings on BMT 3.0.2.

I just scanned that file using 3.1 and it worked fine. I'm not sure if this is a 3.0.x issue or not, but I'd upgrade to the 3.1 release and try it again. How are you scanning it; commandline, web ui, stv, or automatic plugin?

[craigap]

Originally Posted by stuckless

I'm not sure what would cause it to do this... but some things that I would try are....
1. Use Shift+Reload on the browser to force a reload of the webpage.
2. Use another browser to see if it is consistently broken in different browsers
3. Blow away the bmtweb dir (not the .war file) and restart the sagetv server, and try again.

Could you elaborate in much more detail exactly what I need to be doing in #3. I'm not exactly savvy when it comes to navigating the files and identifying/knowing what is what.

Thanks,

Craig

[mkanet]

The great thing about sageTV is if people do frequent software updates, they get a convenient backup of their existing Wiz.bin with a respective time stamp.

Sean, I know the thought has crossed your mind more than once, "why do I even bother with TV support (both series and movies) or the Automatic plugin". These components are what what makes your sageTV addon arguably the most popular addon to sageTV. This is why I tried so hard; over and over again, until I finally got the automatic plugin working. For whatever it's worth, thank you for putting up with people who seem to not understand the amount of work you dedicate to this. Also, thank you for addressing the automatic TV Movies lookup issue. I can't wait to see this working!

-Michael

Originally Posted by stuckless

This is precisely why I was reluctant to add TV support into bmt... I hope you backed up your metadata, because if not, then you hosed. BMT is not a miracle worker, and when it doing fuzzy logic title matching, then there is going to some room for error. The problem in your case is the that bmt selected an exact match from the TVDB for "Jon & Kate Plus 8", but the episode that you wanted was called, "Jon cooks Korean Dinner", not "Korean Dinner", so it didn't do an exact match on the episode. When that happens, bmt still tries to scape some data and bring it back.

Sure, you can argue, that well, bmt should have [insert your personal wish here], but the reality, is that this is one 1 person team, that is stretched pretty thin.

The best thing you can do, to protect yourself again this type hosing, is to make regular backups, and NEVER use the automatic scanner. The web ui has a manual lookup scanner, so you can use that instead of the automatic scanning. Automatic scanning should only be used by people that are OK if they don't 100% correct results (which is me).

From the web ui, you can do manual scans for your jon and kate episodes. For the jon and kate episodes, enter the season and episode #s in the search box, and it will bring down the metadata.

I know how frustrating this is... but bmt cannot, unfortunately, make all the right decisions when it's running as an automated tool.

[stuckless]

Originally Posted by craigap

Could you elaborate in much more detail exactly what I need to be doing in #3. I'm not exactly savvy when it comes to navigating the files and identifying/knowing what is what.

In your SageTV home directory (where you extracted bmt and other plugins), you'll have a jetty/webapps/ directory. Delete the bmtweb dir from that directory, and then restart sage, wait a minute and then try again.

[voorhees]

I am trying to get up to speed with BMT 3.1 (really just BMT in general). What a great tool, thanks so much for your hard work.

I did an automatic scan of everything and got mixed results for what I was looking for concerning TV (as the recent posts mentioned) and some of my movies. So, I reloaded my wiz.bin (I keep 21 days of automatic backups for this reason) and everything is good again.

I have the automatic plugin turned off, and I am going through and changing the movies that it picked incorrectly. The web interface is great. It took a while to figure out what did what, but once you get a handle on it, it is pretty slick. Now I am trying to update the fanart for newly SageTV recorded TV shows. I do not want the metadata updated, just the fanart. If I browse for recordings and search for the metadata to get the fanart it looks like it will update both the metadata and the fanart. If I Scan for the recordings and check nothing in the Metadata/Fanart Options, I only get the recorded movies in my recordings and not the TV shows. So, I am thinking to get the fanart for a SageTV recorded TV show I need to Scan for recordings and only check Update Fanart in the Metadata/Fanart Options. Will that get me my desired results of only pulling down and updating the fanart for a SageTV recorded TV show, and not the metadata?

Thanks.

[craigap]

Originally Posted by stuckless

In your SageTV home directory (where you extracted bmt and other plugins), you'll have a jetty/webapps/ directory. Delete the bmtweb dir from that directory, and then restart sage, wait a minute and then try again.

This appears to have repaired it for now. I have access to the web ui.

Thanks

Craig

[stuckless]

Originally Posted by craigap

This appears to have repaired it for now. I have access to the web ui.

For some unknown reason, some people seem to experience this from time to time. I'm not sure what would cause this happen, since bmt doesn't modify anything in that directory once it's installed.

[razrsharpe]

Originally Posted by voorhees

I do not want the metadata updated, just the fanart. ... So, I am thinking to get the fanart for a SageTV recorded TV show I need to Scan for recordings and only check Update Fanart in the Metadata/Fanart Options.

precisely (check scan recordings and update fanart; uncheck update metadata)

click on the little question mark in that dialog box for more explanation

[craigap]

I don't know if this has been mentioned before, but on web ui my last scan date and Total Scanned never change from Wed Dec. 31 18:00 CST 1969 and Zero, respectively.