Metadata Tools 4.0 Beta Thread (aka BMT 4.0)

[bikesquid]

Originally Posted by stuckless

@bikesquid - it almost seems as if your install is not quite right. I know that you have a very large collection... and I don't know if that is causing some issues or not. When you click on "Sage Imports" does it just show nothing at all? Does the spinning circle start and then stop spinning? If you have DVDs, does the DVD View show anything? You should be able to start over by removing the imports and then adding them back in. You can leave the Recordings, but maybe remove the bulk of your other imports and then start by adding in a small import.

I wouldn't be at all surprised if my install wasn't quite right, is there any way to figure out what's wrong? Jetty seems to be working as sjq is fine....

When I click on "Sage Imports" I get a folder for each drive there are files on. Clicking on one of those brings me to a view of each movie folder, clicking on one of those gets me nothing, regardless of first character missing or not. the icon 'shrinks' to the center and goes away. I think it says "loading" as it fades. Doing this in DVD shows the metadata details so I know it's partially working.


This may sound really like a dumb question, but how exactly do I 'remove' the imports?

[razrsharpe]

Originally Posted by bikesquid

This may sound really like a dumb question, but how exactly do I 'remove' the imports?

Detailed Setup -> General -> Media Center Import Directories

EDIT: To get to the Detailed Setup (from the main screen on SageMC and the Default STV... dont know which stv you are using):
On SageMC: My Menu -> Setup -> SageTV Setup -> Detailed Setup
On the Default STV: Setup -> Detailed Setup

[bikesquid]

Originally Posted by razrsharpe

Detailed Setup -> General -> Media Center Import Directories

You know how when someone tell you an answer and seems so, so obvious?.... Thx for that!

I'd assumed there would be a way within bmt to do that as well... maybe a feature request here for a 'delete' option as well as a find metadata option. I know it would be rescanned on next round, but it would still be useful to nuke individual titles that were problematic.

[stuckless]

Originally Posted by bikesquid

When I click on "Sage Imports" I get a folder for each drive there are files on. Clicking on one of those brings me to a view of each movie folder, clicking on one of those gets me nothing, regardless of first character missing or not. the icon 'shrinks' to the center and goes away. I think it says "loading" as it fades. Doing this in DVD shows the metadata details so I know it's partially working.

I wonder if it's something about your structure... are you using a single folder each each movie? I do this dvds, and they show up fine, but I don't do this for normal .avi movies.

Originally Posted by bikesquid

You know how when someone tell you an answer and seems so, so obvious?.... Thx for that!

I'd assumed there would be a way within bmt to do that as well... maybe a feature request here for a 'delete' option as well as a find metadata option. I know it would be rescanned on next round, but it would still be useful to nuke individual titles that were problematic.

I'll look at adding a delete option.

BTW, you can add remove video import dirs from within the web ui, using Configure -> SageTV -> Video Sources

[bikesquid]

Originally Posted by stuckless

You can leave the Recordings, but maybe remove the bulk of your other imports and then start by adding in a small import.

OK, so Ive reimported a very small drive and all series brought in are still as described: missing the first character and blank when drilled down. Maybe there's something about the directories? To that end, here's one example directory and contained metadata file contents... all pulled from thetvdb via metabrowser. Hopefully there's a clue in here about what's going on.....
:
I:\Palin On Art\
I:\Palin On Art\series.xml
I:\Palin On Art\banner.jpg
I:\Palin On Art\Season 1\
I:\Palin On Art\Season 1\S01E01.avi
I:\Palin On Art\Season 1\S01E02.avi
I:\Palin On Art\Season 1\S01E03.avi
I:\Palin On Art\Season 1\metadata\
I:\Palin On Art\Season 1\metadata\S01E01.xml
I:\Palin On Art\Season 1\metadata\S01E02.xml
I:\Palin On Art\Season 1\metadata\S01E03.xml
I:\Palin On Art\Season 1\metadata\509901.jpg
I:\Palin On Art\Season 1\metadata\509911.jpg
I:\Palin On Art\Season 1\metadata\509921.jpg

I:\Palin On Art\series.xml:
<Series>
<id>88281</id>
<Actors />
<Airs_DayOfWeek />
<Airs_Time />
<ContentRating />
<FirstAired>2008-07-12</FirstAired>
<Genre>|Documentary|</Genre>
<IMDB_ID />
<Language>en</Language>
<Network />
<Overview>Wilhelm Hammershoi, Anne Redpath and the four men known as the Scottish Colourists share one thing. None is as well-known as they deserve to be. What drew me to them all was my admiration and delight in their superb control of colour and their ability to use it to create a dazzling array of moods. There is an air of quiet confidence in all these artists and a determination to produce something fresh and original. They knew what they wanted and they knew it had to be special. They play with light and colour in different ways but always with consummate technical skill, making each picture an eye-opener. </Overview>
<Rating />
<Runtime>60</Runtime>
<SeriesID />
<SeriesName>Palin on Art</SeriesName>
<Status>Ended</Status>
<banner>graphical/88281-g.jpg</banner>
<fanart />
<lastupdated>1266192164</lastupdated>
<zap2it_id />
<DisplayOrder>airdate</DisplayOrder>
<FetchOrder>airdate</FetchOrder>
<LockData>False</LockData>
</Series>

I:\Palin On Art\Season 1\metadata\S01E01.xml:
<Item>
<id>509911</id>
<DVD_chapter />
<DVD_discid />
<DVD_episodenumber />
<DVD_season />
<Director>Eleanor Yul</Director>
<EpisodeName>Redpath</EpisodeName>
<EpisodeNumber>1</EpisodeNumber>
<FirstAired>1997-12-15</FirstAired>
<GuestStars />
<IMDB_ID />
<Language>en</Language>
<Overview>Anne Redpath (1895 – 1965) Born in Galashiels, Anne Redpath studied at Edinburgh College of Art and Moray House from 1913, qualifying as an art teacher in 1917. She married in 1920 and settled in France, painting little until returning to Edinburgh in 1934. Despite almost giving up painting at the start of her career, she went on to produce a superb body of work which was both popular with the public and acclaimed by critics. Redpath was an intriguing and complex character. Although she had a strict congregationalist upbringing in the Scottish Borders, she developed a passion for Catholic architecture and ornament. She travelled the world and had a weakness for Parisian couture, yet she remained a committed Scot with strong left-wing views. These dichotomies in her life were reflected in her work, which combines the sensual with the intellectual and the familiar with the exotic. Her paintings display an exceptional degree of individuality and masterly skill, and her use of colour and contrast, shadow and light confirm her standing as Scotland’s greatest woman painter of the mid 20th century and arguably, one of Europe’s finest artists. This film is the result of a long and fruitful relationship with BBC Scotland, and in particular producer Mhairi McNeill and director Eleanor Yule, who first put me on the road to arts documentaries and have been a constant support and inspiration ever since.</Overview>
<ProductionCode />
<Rating />
<SeasonNumber>1</SeasonNumber>
<Writer />
<absolute_number>1</absolute_number>
<airsafter_season />
<airsbefore_episode />
<airsbefore_season />
<filename>episodes/88281/509911.jpg</filename>
<lastupdated>1266192164</lastupdated>
<seasonid>49321</seasonid>
<seriesid>88281</seriesid>
<LockData>False</LockData>
</Item>

[bikesquid]

Originally Posted by stuckless

I wonder if it's something about your structure... are you using a single folder each each movie? I do this dvds, and they show up fine, but I don't do this for normal .avi movies.

I've been storing movies/tv shows on pc's for a long time, the format has changed over the years, but .avi, .mpg, .ts, .vob, .wmv, .mp4 are all prevelant, and each movie has it's own directory, regardless of source format. (I actually use the directory name to sort and manage rather than the file name, so a directory that contains a movie in two parts will be in the same directory.) In short, yes, a separate directory for each movie. Each tv series is stored in \series name\season #\S##E##. format.

As an example:
2 fast 2 furious is listed as fast 2 furious within bmt and the directory looks like this:
D:\2 fast 2 Furious\folder.jpg
D:\2 fast 2 Furious\backdrop.jpg
D:\2 fast 2 Furious\2_FAST_2_FURIOUS.mp4
D:\2 fast 2 Furious\mymovies.xml

mymovies.xml:
<Title>
<LocalTitle>2 Fast 2 Furious</LocalTitle>
<OriginalTitle>2 Fast 2 Furious</OriginalTitle>
<SortTitle>2 Fast 2 Furious</SortTitle>
<Added>8/27/2009 6:20:21 PM</Added>
<ProductionYear>2003</ProductionYear>
<RunningTime>101</RunningTime>
<IMDBrating>5.1</IMDBrating>
<MPAARating>PG-13</MPAARating>
<Description>Former cop, Brian O'Conner is finally arrested after letting his leader escape the law. To avoid the consequences, he must now work with an old college friend and help the police arrest a local drug exporter. How Fast Do You Want It?</Description>
<Type />
<AspectRatio>2.35:1</AspectRatio>
<LockData>False</LockData>
<IMDB>tt0322259</IMDB>
<TMDbId>584</TMDbId>
<Genres>
<Genre>Action</Genre>
<Genre>Crime</Genre>
<Genre>Road Movie</Genre>
</Genres>
<Persons>
<Person>
<Name>John Singleton</Name>
<Type>Director</Type>
<Role />
</Person>
<Person>
<Name>Paul Walker</Name>
<Type>Actor</Type>
<Role>Brian O'Conner</Role>
</Person>
<Person>
<Name>Tyrese Gibson</Name>
<Type>Actor</Type>
<Role>Roman Pearce</Role>
</Person>
<Person>
<Name>Eva Mendes</Name>
<Type>Actor</Type>
<Role>Monica Fuentes</Role>
</Person>
<Person>
<Name>Cole Hauser</Name>
<Type>Actor</Type>
<Role>Carter Verone</Role>
</Person>
<Person>
<Name>Ludacris</Name>
<Type>Actor</Type>
<Role>Tej</Role>
</Person>
<Person>
<Name>Thom Barry</Name>
<Type>Actor</Type>
<Role>Agent Bilkins</Role>
</Person>
<Person>
<Name>James Remar</Name>
<Type>Actor</Type>
<Role>Agent Markham</Role>
</Person>
<Person>
<Name>Devon Aoki</Name>
<Type>Actor</Type>
<Role>Suki</Role>
</Person>
<Person>
<Name>Amaury Nolasco</Name>
<Type>Actor</Type>
<Role>Orange Julius</Role>
</Person>
<Person>
<Name>Michael Ealy</Name>
<Type>Actor</Type>
<Role>Slap Jack</Role>
</Person>
<Person>
<Name>Jin Auyeung</Name>
<Type>Actor</Type>
<Role>Jimmy</Role>
</Person>
<Person>
<Name>Edward Finlay</Name>
<Type>Actor</Type>
<Role>Agent Dunn</Role>
</Person>
<Person>
<Name>Mark Boone Junior</Name>
<Type>Actor</Type>
<Role>Detective Whitworth</Role>
</Person>
<Person>
<Name>Matt Gallini</Name>
<Type>Actor</Type>
<Role>Enrique</Role>
</Person>
</Persons>
<Studios>
<Studio>Universal Pictures</Studio>
<Studio>Mikona Productions GmbH &amp; Co. KG</Studio>
<Studio>Original Film</Studio>
</Studios>
</Title>

[stuckless]

Here's beta8

Issues Fixes
1. Fixed issue in automatic plugin where failed items were not being handled correctly
2. Fixed bug in bmt api IsMetadataScanRunning()
3. Fixed bug in MetataProcess; wasn't notifying the monitor that it had done some work. (GetMetadataScanComplete())
4. Adjusted the SystemMessage for failed scans to be more readable.
5. Added CachCleanUpTask for periodically cleaning up the URL cache

[stuckless]

@bikesquid - Thanks for uploading that info. When I get a chance, I will test using that structure and see what happens. (hopefully within the next day or so)

[heffe2001]

Not sure if this has been mentioned, but in the older versions, you could click the status link at the top, and it would refresh the screen with the updated status. In the b7 version released yesterday, clicking on the status link at the top doesn't appear to do anything at all.

[skiingwiz]

This is from beta 7. My recording of tonight's episode of "24" got the wrong episode information. It chose Season 1 Episode 1 instead of Season 8 Episode 9. My guess is this is because the two episode titles are so similar. (The former is "Day 1: 12:00 AM - 1:00 AM" and the latter is "Day 8: 12:00 AM - 1:00 AM". They only differ by 1 character.)

I didn't have this problem in 3.x. This would be pretty easy to solve if BMT had access to the original air date. Is GetOrginalAiringDate() one of the Sage API calls that can't be used because it causes Sage to call the automatic plugin again (causing an infinite loop).

The filename of the recording is "24-Day81200AM100AM-6608151-0.mpg" if that helps. My Good Score Threshold is at the default of 0.9. (It always has been) Is there any other information that would be useful?

[stuckless]

Originally Posted by heffe2001

Not sure if this has been mentioned, but in the older versions, you could click the status link at the top, and it would refresh the screen with the updated status. In the b7 version released yesterday, clicking on the status link at the top doesn't appear to do anything at all.

Added it to the list... should be easy enough to do.

Originally Posted by skiingwiz

This is from beta 7. My recording of tonight's episode of "24" got the wrong episode information. It chose Season 1 Episode 1 instead of Season 8 Episode 9. My guess is this is because the two episode titles are so similar. (The former is "Day 1: 12:00 AM - 1:00 AM" and the latter is "Day 8: 12:00 AM - 1:00 AM". They only differ by 1 character.)

I didn't have this problem in 3.x.

I don't this engine changes in 4.0... I'm guessing that 3.1 had the same issue, it's just that you didn't encounter such as close match before. I'll check out 3.1 as well, and see if I get the same results, if I don't then I can track down the change.

This would be pretty easy to solve if BMT had access to the original air date. Is GetOrginalAiringDate() one of the Sage API calls that can't be used because it causes Sage to call the automatic plugin again (causing an infinite loop).

Currently the tv scanner can lookup by date, but it tries a title match first, and then date. I'm wondering if I should try date first and then title??

The filename of the recording is "24-Day81200AM100AM-6608151-0.mpg" if that helps. My Good Score Threshold is at the default of 0.9. (It always has been) Is there any other information that would be useful?

Thanks for the filename... that'll help me track it down.

[razrsharpe]

Originally Posted by stuckless

I'm wondering if I should try date first and then title??

Well which is more likely to be correct, original air date or title? In an ideal world i would say do both independently, compare the results, and only return something if they match (or have a configurable preference for which should win)

[stuckless]

Originally Posted by razrsharpe

Well which is more likely to be correct, original air date or title? In an ideal world i would say do both independently, compare the results, and only return something if they match (or have a configurable preference for which should win)

That the problem... if bmt is fired off as a result of a new recording, then the date is probably good. But, if this is a result of importing an older airing, then the title is more likely to be correct.

I'll do some testing... and see if I can come up with a better approach. There's no issue in doing both searches, but, if I got a "good hit" on the title, then it would never do the second search by date, or vice versa.

[evilpenguin]

Originally Posted by stuckless

That the problem... if bmt is fired off as a result of a new recording, then the date is probably good. But, if this is a result of importing an older airing, then the title is more likely to be correct.

I'll do some testing... and see if I can come up with a better approach. There's no issue in doing both searches, but, if I got a "good hit" on the title, then it would never do the second search by date, or vice versa.

FWIW, I've been using original airdate in Mediascaper for a while and its worked great on SageTV recordings. My only two failure cases were...

• If a show came down with generic EPG data and the original airdate would be set to the first episode of the series (Happens occasionally with syndicated shows). This was easily detectable though because those airings would have a series title but not an episode title. I was never quite sure how to handle these so I just left the original airdate, season #, episode #, and episode name blank.
• If a show came down with incorrect airing date. This happened a lot of Toon Disney where-in they just had completely incorrect date data for cartoons from the 80's and 90's. But this was easily detectable too as the date would fail to return a result and, even though I didn't implement it, a title fall back search would have recovered.

[stuckless]

Originally Posted by evilpenguin

FWIW, I've been using original airdate in Mediascaper for a while and its worked great on SageTV recordings. My only two failure cases were...

• If a show came down with generic EPG data and the original airdate would be set to the first episode of the series (Happens occasionally with syndicated shows). This was easily detectable though because those airings would have a series title but not an episode title. I was never quite sure how to handle these so I just left the original airdate, season #, episode #, and episode name blank.
• If a show came down with incorrect airing date. This happened a lot of Toon Disney where-in they just had completely incorrect date data for cartoons from the 80's and 90's. But this was easily detectable too as the date would fail to return a result and, even though I didn't implement it, a title fall back search would have recovered.

Thanks EP I'll look at doing that, with. A title fallback.

[skiingwiz]

Originally Posted by stuckless

I don't this engine changes in 4.0... I'm guessing that 3.1 had the same issue, it's just that you didn't encounter such as close match before. I'll check out 3.1 as well, and see if I get the same results, if I don't then I can track down the change.

I can definitely confirm that 3.1 didn't have this issue with "24" for me. This show is probably the worst case possible for matching by title. Every season has the same 24 episode names, with only the day number changed. So for every episode there are 8 matches that are only 1 character different (since this is the 8th season.)

Originally Posted by stuckless

Currently the tv scanner can lookup by date, but it tries a title match first, and then date. I'm wondering if I should try date first and then title??

Originally Posted by stuckless

Thanks EP I'll look at doing that, with. A title fallback.

I think this is probably the best approach. It is a rare case that a show has more than one episode on any day, so original air date ought to get to the match right away.

Thanks for all the work on BMT. It is an essential part of my Sage setup

[razrsharpe]

Originally Posted by skiingwiz

It is a rare case that a show has more than one episode on any day, so original air date ought to get to the match right away.

not as rare as you might think... just looking at 24 season 8.. episodes 1 & 2, 3 & 4, and 23 & 24 all air on the same day... lots of shows air multiple episodes for the season premier or finale... also sometimes they'll routinely air 2 1/2 hr shows on the same day....

[evilpenguin]

Originally Posted by razrsharpe

not as rare as you might think... just looking at 24 season 8.. episodes 1 & 2, 3 & 4, and 23 & 24 all air on the same day... lots of shows air multiple episodes for the season premier or finale... also sometimes they'll routinely air 2 1/2 hr shows on the same day....

Ahh, yeah, good call, those were a pain.

If your EPG actually does split them into two separate airings they're easy to deal with as you can detect the airdate collision and fall back to a title search. But there's also, a far more annoying case where your EPG will combine both episodes into single airing. For example this was my 24 premire from last season...

24-Day7600PM700PMDay7700PM800PM-3788655-0.mpg

which pretty much impossible to deal with automatically. I ended up just letting it scrape by airdate to get the first episode's info and then manually modified the info to be for both episodes

Oh, and while i'm bitching about how hard 24 is to scrape, I also had a case where my EPG came down with this...

24-Day7Midnight100AM-4186040-0.mpg

which would be impossible to scrape without using airdates because the official episode name was...

Day 7: 12:00 AM - 1:00 AM

[stanger89]

Originally Posted by bikesquid

OK, so Ive reimported a very small drive and all series brought in are still as described: missing the first character and blank when drilled down. Maybe there's something about the directories? To that end, here's one example directory and contained metadata file contents... all pulled from thetvdb via metabrowser. Hopefully there's a clue in here about what's going on.....

Sean,

There's something going on when using drives and not UNC paths. I'm seeing the same thing when I import "S:" or "S:\" as an import directory. But if I change it to the UNC path "\\server\TV" then I can browse just fine.

[stuckless]

Originally Posted by stanger89

Sean,

There's something going on when using drives and not UNC paths. I'm seeing the same thing when I import "S:" or "S:\" as an import directory. But if I change it to the UNC path "\\server\TV" then I can browse just fine.

You mean, just a drive, S:, an dnot a path on the that drive? Is is they a dvd drive? (just curious)... This will be tricky for me to test, but I'll give it a shot.

Thanks,
Sean.