Metadata Tools 4.0 Beta Thread (aka BMT 4.0)

[skiingwiz]

Originally Posted by stuckless

The context hasn't changed from 3.x to 4.x... and I don't actually know what needs to happen in order to enable ssl. Has anyone used 4.x with ssl?

It works fine for me under SSL. I had it working that way with 3.x and didn't have to change anything for 4.x.

[jonnydeath]

Originally Posted by skiingwiz

It works fine for me under SSL. I had it working that way with 3.x and didn't have to change anything for 4.x.

It works now! That was weird. It would not get the status for anything under ssl the first day (even restarted the server a few times). After logging in a few times without SSL, it now works fine with SSL. Weird. The only thing I did was clean out a bunch of duplicate jars. Thanks for the replys

[stuckless]

Here's beta15

Fixed/Added
1. Added ability to provide custom vfs xml files (vfs xml also validates against the dtd)
2. changed overwrite metadata settings (overwrite will now overwrite if not exist) (this basically undoes the earlier changed, but I also added some extra checks, etc. Should work better)
3. added tools menu (clean properties dialog, create support request dialog)
4. fixed tvdb language issue (can now search using other languages)
5. fixed issue in the provider selection, for searching, where the selected provider would not be used.
6. Metadata Scans now build up the collections to scan before scanning, so that we can get an accurate % complete.
7. Add new views; "Missing Metadata", "By Year"
8. Minor UI updates. Shows % scanned; incons for remove/cancel buttons;

If you are wondering how we got to 15... don't worry, there were a few other debug builds that I did for various reaons, but they were not really beta builds.

Some things to note in this build...
1. I think I have worked out the "overwrite" properties. Hopefully this should now behave as people expect. Bottom line is the if overwrite is enabled then existing metadata will be overwritten. If it's not checked, then metadata will not be overwritten, but it will still be fetched if it does not exist.

2. I added a couple new views. The Missing Metadata view will find all video items that are missing MediaTitle and MediaType values, which is essential for fanart.

3. I added a "Tools" menu. My goal here, is to throw misc actions in there. Right now I have 2 actions. Create a Support Zip wiil basically enable you to create a zip containing your log files and even a structure layout of your videos for support purposes. You control what you want to go into the zip. The other tool is a .properties cleaner. If you use it, it will remove all .properties that are in your video import paths. ie, it cleans all files that end in .properties.

So, if you have ideas for the "tools" menu, let me know... If they are easy to implement, then I'll add them as time permits.

If you are curious about item #1 in the Fixed/Added list... I'll touch more on the VFS (virtual filesystem) stuff later... but for now, it means that I've added the ability (in phoenix, which is used by bmt) to provide your own vfs xml files that will not be overwritten by reinstalls. The VFS is what controls the "Views" in bmt.

[Tiki]

Originally Posted by stuckless

Here's beta15

1. I think I have worked out the "overwrite" properties. Hopefully this should now behave as people expect. Bottom line is the if overwrite is enabled then existing metadata will be overwritten. If it's not checked, then metadata will not be overwritten, but it will still be fetched if it does not exist.

So, two questions:
1- will the overwrite fanart behave the same way (ie if "true" you will replace existing fanart, if "false" you will only add new or missing fanart)?
2- should we expect the overwrite metadata to work similar to previous builds in that regardless of setting, it will never update core metadata for Sage Recorded TV, but will always update some of the extended metadata?

[Tiki]

After installing Beta 15 I got the following Sage System Message:

Error - General System Message
Error in VFS File: .STVs\Phoenix\vfs\ext\vfs.xml; Document is invalid: no grammar found.

Any idea what this means? I see that this file does exist in my installation. Here is what it currently contains:

<vfs>
    <!-- 
    <views>
        <view id="videobyyear" label="Videos (by Year)" flat="true">
            <description>Show Videos grouped by year</description>
            <source name="dvd" />
            <source name="video" />
            <source name="bluray" />
             
            <children>
                <group by="year" />
                <sort by="title" />
            </children>
        </view>

        <view id="sageimports" visible="false"/>
    </views>
     -->
</vfs>

[zovirax_tomer]

i have two problems:

1. unable to download information for hebrew title
error message:
Unable to get Metadata

2. it stop to import automatically.
if for example i have a new episode of LOST, BMT don't create a .properties file
and download the FANART.

thank you!

[stuckless]

Originally Posted by Tiki

So, two questions:
1- will the overwrite fanart behave the same way (ie if "true" you will replace existing fanart, if "false" you will only add new or missing fanart)?
2- should we expect the overwrite metadata to work similar to previous builds in that regardless of setting, it will never update core metadata for Sage Recorded TV, but will always update some of the extended metadata?

True will replace exising fanart, false will only add new fanart.
For metadata it's alot more complicated..
1. for sage recorded TV shows, I never update the core metadata
2. if overwrite metadata/fanart is true, then the custom metadata fields get overwritten (ie, mediatype, mediatype, etc) since they are required for fanart.
3. if overwrite is true, the .properties files are overwritten. if overwrite is false, then .properties files are not overwritten, but new ones are created,if they don't exist.

Originally Posted by Tiki

After installing Beta 15 I got the following Sage System Message:

You can delete that file. It was shipped in build 13/14 when I first started to work on breaking out the vfs stuff... right now there should only be 1 xml file in the vfs/ext dir, and that is bmt.xml.

Originally Posted by zovirax_tomer

i have two problems:

1. unable to download information for hebrew title
error message:
Unable to get Metadata

2. it stop to import automatically.
if for example i have a new episode of LOST, BMT don't create a .properties file
and download the FANART.

Could you try doing metadata lookups based on season/episode #, and see if that works. Also, I was having some issues with tvdb, so I couldn't fully test it. It may be that doing lookups by episode title may not work, but I'll do some testing.

[zovirax_tomer]

yes.
with season/episode number it working.
but no FANART...

[stuckless]

language shoudl not affect fanart... but it will affect how episode titles are matched. On the status screen check to see that fanart is enabled and that the Fanart Central Folder doesn't show up in red. If it's red, then it means that your fanart folder does not exist... and lastly in the Configuration check that fanart is enabled (you can just search for "fanart" in the configuration screen).

[petereader]

Sean,
Just wanted to update you on this: it appears that my cross-contamination issue has disappeared. Not sure exactly what did it, I updated from beta9 to 11 and removed those dup jars we talked about (the ones you said shouldn't cause an issue but to remove them anyways). I'll be updating to 15 tonight, but thought you may want to know this.

I know you've been fielding these questions quite a bit lately, but I just wanted to be clear: let's say I have a variety of tv shows that have descriptions already, but I'd like them to get season/episode number that they do not currently have using the automatic plugin (but keep the current descriptions from the EPG). Should I be selecting overwrite metadata if I'm using the new beta(15)?

[razrsharpe]

Originally Posted by petereader

I know you've been fielding these questions quite a bit lately, but I just wanted to be clear: let's say I have a variety of tv shows that have descriptions already, but I'd like them to get season/episode number that they do not currently have using the automatic plugin (but keep the current descriptions from the EPG). Should I be selecting overwrite metadata if I'm using the new beta(15)?

My reading of beta 15's release notes is that overwrite metadata should not be checked if you want to maintain the description, etc from the epg but still fetch the other metadata... season/episode info for example

Originally Posted by stuckless

1. I think I have worked out the "overwrite" properties. Hopefully this should now behave as people expect. Bottom line is the if overwrite is enabled then existing metadata will be overwritten. If it's not checked, then metadata will not be overwritten, but it will still be fetched if it does not exist.

[petereader]

Originally Posted by razrsharpe

My reading of beta 15's release notes is that overwrite metadata should not be checked if you want to maintain the description, etc from the epg but still fetch the other metadata... season/episode info for example

Thanks razr,.

I had read that differently, but now that you clarified that, I don't know why I was reading that way to begin with

[skiingwiz]

Originally Posted by razrsharpe

My reading of beta 15's release notes is that overwrite metadata should not be checked if you want to maintain the description, etc from the epg but still fetch the other metadata... season/episode info for example

I don't think BMT will overwrite the EPG info, no mater what the overwrite setting is. See #1 below in stuckless's post.

Originally Posted by stuckless

For metadata it's alot more complicated..
1. for sage recorded TV shows, I never update the core metadata
2. if overwrite metadata/fanart is true, then the custom metadata fields get overwritten (ie, mediatype, mediatype, etc) since they are required for fanart.
3. if overwrite is true, the .properties files are overwritten. if overwrite is false, then .properties files are not overwritten, but new ones are created,if they don't exist.

[bikesquid]

updated to 4.15 and still not getting .properties after a rescan. I've got sageTV/Metadata Properties/Update .properties when changed checked. I'm just trying to get recorded tv show to update/create the .properties so have removed all video source directories. what else do I need to do? Not trying to get automatic scanning working yet....

EDIT: Never mind, found the problem. thx.

[zovirax_tomer]

Originally Posted by stuckless

language shoudl not affect fanart... but it will affect how episode titles are matched. On the status screen check to see that fanart is enabled and that the Fanart Central Folder doesn't show up in red. If it's red, then it means that your fanart folder does not exist... and lastly in the Configuration check that fanart is enabled (you can just search for "fanart" in the configuration screen).

all my settings is ok.
but still, not import automatically information for new video file.

this line is red: JVM Max Memory
(i upload a picture of the status screen)


i look in the log: mailcheck.log
and i get error:

2010-03-01 13:00:43,924 ERROR [Ministry] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files\SageTV\SageTV\.\STVs\Phoenix\Configuration\Phoenix.xml
2010-03-01 13:00:46,155 ERROR [AWT-EventQueue-0] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files\SageTV\SageTV\.\STVs\Phoenix\Configuration\Phoenix.xml

[skiingwiz]

Originally Posted by zovirax_tomer

all my settings is ok.
but still, not import automatically information for new video file.

this line is red: JVM Max Memory
(i upload a picture of the status screen)


i look in the log: mailcheck.log
and i get error:

2010-03-01 13:00:43,924 ERROR [Ministry] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files\SageTV\SageTV\.\STVs\Phoenix\Configuration\Phoenix.xml
2010-03-01 13:00:46,155 ERROR [AWT-EventQueue-0] sagex.phoenix.configuration.XmlMetadataProvider (XmlMetadataProvider.java:20) - Missing Configuration File: C:\Program Files\SageTV\SageTV\.\STVs\Phoenix\Configuration\Phoenix.xml

The problem in the log file can be ignored. As for the JVM Memory being red, that itself isn't a problem since BMT just colors it below a certain value (768MB, I think). But you do have a system message there complaining about an OutOfMemoryError. You should definitely fix that problem before worrying about anything else.

EDIT: On closer inspection, what language is that show title? Is it a language that reads right-to-left? Is it possible that Sage is putting the show name at the end of the file name rather than the beginning because of that? That would be a problem for the BMT filename scraper. (Also possibly a bug in Sage)

[zovirax_tomer]

hi skiingwiz

i install clean sagetv
delete all the properties files from the media Folder (movies and tvshow)
but the bmt dont import any data for these files.
i do a scan to the library (Reindex).
the System Messages is:
Metadata Scan Completed. Updated: 357; Failed: 0; Skipped: 0




edit:
from Browse the import is working good although for hebrew title (including FANART).

[stuckless]

@zovirax_tomer - if you a running the beta15, then try building a support zip of just your log files (Tools -> Create Support Request) and email them metadatatools at gmail dot com. It's odd that it would report 100% success and NOT import any metadata... perhaps the log files will shed some light.

[zovirax_tomer]

i dont know how, it start to work. (after install and uninstall)
now the Metadata and FANART is downloading in Hebrew for series.
but there is a problem:
same of the series that download the info from thetvdb.com have a format problem:
in Show Title: àáåãéí
this is not hebrew.
and the Fanart Location:
C:\Program Files\SageTV\SageTV\STVs\Phoenix\Fanart\TV\àáåãéí\Season 1
but in windows folder the name is ok.

[petereader]

Hi Sean,

I seem to be having a problem getting the automatic plugin to create .properties files. Using the webui works just fine, but I've got a few recordings and imported videos that should be getting .properties files generated by the automatic plugin and are not, despite rescanning my import library a few times. It appears as though the plugin is scanning becasue I get system messages with no fail, but it doesn't appear as though anything updated. I noticed the log files have quite a few entries that look like this:

2010-03-06 16:56:27,663 INFO [main-MediaFileHandler] org.jdna.process.MetadataProcessor$1 (MetadataProcessor.java:148) - Resource: SageMediaFile:[MediaFile[id=5205620 A[5244477,5205621,"Battlestar Galactica Season 3/Battlestar Galactica S03E19",0@0304.08:26,42] host=presidente encodedBy= format=AVI 0:42:43 2282 kbps [#0 Video[MPEG4-Video 25.0 fps 960x528 20:11 progressive]#1 Audio[AC3 48000 Hz 6 channels 384 kbps idx=1]] \\Presidente\MoreMovies\Battlestar Galactica Season 3\Battlestar Galactica S03E19.avi, Seg0[Thu 3/4 8:26:27.654-Thu 3/4 9:09:11.134]]] was not accepted by filter.

The log file is too big to post here (wow! they fill up fast), so I'm using the webui to zip it up and mail it to you in case there's more valuable information in there.

Any idea why this is happening?

Thank you.