Metadata Tools 4.0 Beta Thread (aka BMT 4.0)

[stuckless]

Here's another build...

Fixes/Enhancements
1. Added Sage properties viewer to the configuration
2. Added Regex Editor for regex types
3. Added Fanart dialog for setting fanart, and adding new pieces of fanart. To Change fanart click on the poster icon when in metadata editor.
4. Updated Log4j editor to allow for for fine grained control over logging.
5. Updated support creator tool to include more information about library versions, etc.
6. Fixed bug in the Cached Url Cleanup task... was running too often.
7. Made a small change to the Properties persistence so that it does not load properties before saving them.
8. Added simple log viewer to the log4j panels

So, there are more "enhancements" than fixes in this build. I did make a couple of small changes that I'm hoping may have effect on the cross linked metadata issues.

I added some debug code that I can use for the "missing" media items problems. It's disabled by default, but if you want to enable it and send me the logs, then go for it. To enabled the debugging, go to "Configuration -> Log4j Logging -> Logging" and then select "phoenix" from the drop down. Then change this value, log4j.logger.phoenix.vfs from info to debug and Save. Then browse to your recordings, and then build a support log to send me. You can then change the setting back. This is an expensive logging operation, but what it does is dump ALL the media items that go into making up a view before and after the view is created. I'm hoping it will show me something useful.

#3 above is a simple Fanart dialog that I added. When yuou are in the metadata editor, and you click the thumbnail icon, it will pop open a dialog that will show you the current posters, banners, backgrounds for the item. You can also add/remove items as well. It's a work in progress.

Good luck, and thanks for testing.

[aedgington]

That's cool, it's beta tax Just wanted to make sure it wasn't user error.

For troubleshooting purposes, it happened to me after importing videos to replace TV episodes, applying the metadata & marking them as Sage recordings. Hope that helps narrow it down.

[stuckless]

Well... after much debugging... log sending... etc, I was able to reproduce the issue of the missing media items on my setup... and I think that I've fixed it Here's beta 20.

Issues Fixed
1. Fixed issue where some views were missing items.

Warning Technical non-sense follows... Read at your risk...
This issue came about because I moved some of the internal vfs code to use a java Set instead of List because i didn't want duplicate items in my views. But, as it turns out, in java when you use a Set and your provider your own comparator to identify how 2 items in the set compare, then returning 0 is basically telling the set that 2 items are equal, and the set will not add the second item. I had a bug in the Start Time comparator where it would always return 0 when 2 folders were being compared, and this causes the Set to not allow any more than grouped child.

Bottom line... when you are using Sets... make sure your Comparator never returns 0 unless the 2 items that you are comparing are in fact equal

Thanks to everyone that sent logs... it helped a lot. BTW, if you have the debug logging enabled for phoenix.vfs, then set it back to "warn" since it will have a very large negative impact on the view creation. I've left the debug logging in there, in case I need it again, but it's only dumped if that setting is set to true.

[Wade]

You're the man! My setup is showing all six seasons of "The Office" now.

Thanks,
Wade

[bikesquid]

Originally Posted by stuckless

Issues Fixed
1. Fixed issue where some views were missing items.

I hate to say it... I really do... updated to beta20 and I'm still seeing in 'recordings' only one episode where several exist, they do show up correctly under 'all tv'.

More importantly (to me anyway) I'm still not getting .properties from the automatic plugin.

[bialio]

Two shows of mine are still 'missing' episodes in the All TV (Show/Season) view - 24 only shows a single episode, where there are 8 or so. And I have two Smallville on disk right now and only one shows up there.

The Recordings view doesn't have any folders in it - shows a single episode of every recording I have - seemingly the oldest by file date.

btl.

[bikesquid]

Originally Posted by bialio

Two shows of mine are still 'missing' episodes in the All TV (Show/Season) view - 24 only shows a single episode, where there are 8 or so. And I have two Smallville on disk right now and only one shows up there.

The Recordings view doesn't have any folders in it - shows a single episode of every recording I have - seemingly the oldest by file date.

btl.

are you getting .properties files in your Recorded TV folder?

[stuckless]

Originally Posted by Wade

You're the man! My setup is showing all six seasons of "The Office" now.

At least 1 person is happy

Originally Posted by bikesquid

I hate to say it... I really do... updated to beta20 and I'm still seeing in 'recordings' only one episode where several exist, they do show up correctly under 'all tv'.

Originally Posted by bialio

Two shows of mine are still 'missing' episodes in the All TV (Show/Season) view - 24 only shows a single episode, where there are 8 or so. And I have two Smallville on disk right now and only one shows up there.

The Recordings view doesn't have any folders in it - shows a single episode of every recording I have - seemingly the oldest by file date.

The good news, is that I'm pretty sure that this is related to the "Set" issue... bialio could you enable the debug vfs logging (i posted intructions a few posts back) and then send me the logs/phoenix.log file... maybe your setup will show me something that I'm not seeing the other logs that I've been sent.

Originally Posted by bikesquid

are you getting .properties files in your Recorded TV folder?

I can assure you that I'm seeing .properties.... I added 2 new movies tonight and both of those got scanned, which resulted in .properties.

Are you not seeing .properties for new movies that you add to your collection or just recordings? btw... when i add new movies, I tend to tell sage to scan for new media, using Refresh -> Scan for new media.

[bikesquid]

Originally Posted by stuckless

Are you not seeing .properties for new movies that you add to your collection or just recordings? btw... when i add new movies, I tend to tell sage to scan for new media, using Refresh -> Scan for new media.

I haven't been adding any new movies, just what sage records. I have been doing a refresh -> scan for new media as well as a reindex in hopes that lots of button pushing and generally flailing will help the situation... to date, no luck.

Just for giggles I added a movie (vob file) and while it shows up in imports there is no .properties after a rescan. If I update metadata and find it manually and select save and check the create .properties, a file does in fact appear... same as with sage recordings.

[bialio]

Here's the log file.

[stuckless]

Here's build 21...

Issues Fixed
1. Removed the use of Sets because it caused too many side effects with grouping items with the same name.
2. Added ability to clean up the system messages.

Ok... I've removed the use of the Java Set. I figured I could have made it work, but after further investigation, I'm not sure I like how the java Set treats equals() vs compareTo().

[bikesquid]

Cool! Now the views show all the right things in all the right places.... Still not getting .properties files and have no idea why. Is anyone else having trouble with this or is it just me??


Thanks for getting that view issue resolved!

Doing a 'update metadata' does in fact create .properties if I select that check box.... for all recordings at once, it's just the automatic plug-in that's not happening....

B.

[toricred]

beta 21 is great! This has resolved all my issues and actually seems to be picking up difficult series' metadata easier, although that's probably something at tvdb or my imagination.

[toricred]

OK, so I still have some issues with specific titles that are difficult for BMT4. One example is the Fox series 'Til Death. I'm trying to update the mediatitles.xml file, but I clearly don't understand it. I read the documentation on the wiki and I don't find something that seems to match. I want it to be general for all files that begin with TilDeath-, but not in a directory. Any tutorials would be appreciated.

[stuckless]

Originally Posted by toricred

OK, so I still have some issues with specific titles that are difficult for BMT4. One example is the Fox series 'Til Death. I'm trying to update the mediatitles.xml file, but I clearly don't understand it. I read the documentation on the wiki and I don't find something that seems to match. I want it to be general for all files that begin with TilDeath-, but not in a directory. Any tutorials would be appreciated.

Could you post some filenames... and I'll try to see if I can quickly create a matcher.

[toricred]

TilDeath-AllysPregnant-2058855-0.mpg
CSIMiami-AshestoAshes-2065389-0.mpg

I have other series I can't think of at the moment, but hopefully your help with these will teach me what I need to add my own examples in the future. I don't want to keep bothering you with these kind of requests in the future.

[stuckless]

Originally Posted by toricred

TilDeath-AllysPregnant-2058855-0.mpg
CSIMiami-AshestoAshes-2065389-0.mpg

I have other series I can't think of at the moment, but hopefully your help with these will teach me what I need to add my own examples in the future. I don't want to keep bothering you with these kind of requests in the future.

I haven't tested these... but I would think that the following should work...

	<match>
		<regex>[\\/]TilDeath-</regex>
		<title>'Til Death</title>
		<metadata type="tv" name="tvdb">79384</metadata>
	</match>

	<match>
		<regex>[\\/]CSIMiami-</regex>
		<title>CSI: Miami</title>
		<metadata type="tv" name="tvdb">78310</metadata>
	</match>

So these need to be in a file called MediaTitles.xml in the scrapers folder (There's a MediaTitles.xml.sample there now, you can rename to MediaTitles.xml). And when you changed the MediaTitles.xml, you need to restart sagetv.... I'll look at adding a simple editor to the WebUI for this.

[tmiranda]

Sean,

For a guy who is supposed to be resting at home you sure are working a lot! Take it easy for Pete's sake...

Tom

[stuckless]

Originally Posted by tmiranda

Sean,

For a guy who is supposed to be resting at home you sure are working a lot! Take it easy for Pete's sake...

Tom

I'm actually feeling remarkably well... I get tired easily... My day consists of watching tv and playing around with this

[Skirge01]

I upgraded to beta 21 and most of the missing shows seem to be back, but I haven't looked deep enough to say that they're all there. I'll let you know about that part.

However, I attempted to add a folder to the exclusion list and I'm not sure if I did it right. First I tried adding "\\Mediaserver\Other Videos", which gave me this:

java.util.regex.PatternSyntaxException: Illegal/unsupported escape squence near index 14
\\Mediaserver\Other Videos
              ^
	at java.util.regex.Pattern.error(Unknown Source)
	at java.util.regex.Pattern.escape(Unknown Source)
	at java.util.regex.Pattern.atom(Unknown Source)
	at java.util.regex.Pattern.sequence(Unknown Source)
	at java.util.regex.Pattern.expr(Unknown Source)
	at java.util.regex.Pattern.compile(Unknown Source)
	at java.util.regex.Pattern.<init>(Unknown Source)
	at java.util.regex.Pattern.compile(Unknown Source)
	at org.jdna.bmt.web.server.PreferencesServiceImpl.validateRegex(PreferencesServiceImpl.java:275)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:562)
	at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:188)
	at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:224)
	at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:389)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.handler.rewrite.RewriteHandler.handle(RewriteHandler.java:230)
	at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:879)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
	at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:636)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)

So, I tried "\\\\Mediaserver\\Other Videos", but the only feedback I get from BMT is that "Regex is valid, but it did not match anything." That "test" result doesn't mean much to me. Is it saying that the folder IS a valid path, but there's no media in it? Is it saying that the path SEEMS to be valid, but it can't find it? Or, is it saying something else? It's a little too vague for me to understand. I'm sure it makes perfect sense to you, but it threw me for a loop.

Two more questions (for now, hehe):

Should multiple exclusion paths be separated by commas, semi-colons, or pipes?
If I want the subfolders within an excluded folder to also be excluded, do I need to enter them all or do subfolders inherit the exclusion from the parent?