Utility: Batch Metadata Importer 3.x

[Nick_W]

Originally Posted by jaminben

I've not tried the latest version yet but did you re-import the tool, thats if you've just upgraded to the latest SageMC.

Hi Ben,

Yes I did, that's why it's working

Cheers,

Nick.

[tmiranda]

Originally Posted by stuckless

I'll have to do some tests to see if the plugin is call for Sage Recordings when they are first created.

--fanartOnly is not implmented for the plugin/ondemand functinality. I can look at adding an option for that if you want.

Thanks.

[stuckless]

Originally Posted by Nick_W

Sean,

Sorry if I've missed this, but should the options still be there under SageMC>Extras?

Mine are no longer there, but the automatic fetching is still working great! Have you removed them, or is it me?

TIA,

Nick.

The options under SageMC>Extras should still be there. I haven't purposely removed them and i also haven't done any work on it i first released it. I would try to re-import it again, and see if it shows up.

[Nick_W]

Originally Posted by stuckless

The options under SageMC>Extras should still be there. I haven't purposely removed them and i also haven't done any work on it i first released it. I would try to re-import it again, and see if it shows up.

I imported it again as the first of the imports, and it's there now, thanks

I still get those small buttons though!

Regards,

Nick.

[toricred]

OK, so I'm still having some problems getting the season and episode information to show up. I've discovered that if I manually search for the metadata in Sage it works fine. If I run the search from the command line it doesn't come up.

In order to do more troubleshooting I'd like to understand the flow of the metada. When you search for metadata from the Sage GUI does it push the custom metadata into the Sage DB right away? If done from the command-line does something need to happen to push the metadata in?

[stuckless]

The ondemand (manual) search from within sage is slightly different. It will create the metadata.properties, but it also will use the new SetMediaFileMetadata() set the custom metadata fields.

When you run from the commandline it will create the .properties files that sage will read directly.

When you run as an automatic plugin, it will create the .properties, and read them back in and pass directly to sage when sage asks for the metadata of the item. In the commandline mode, and the automatic plugin, sage won't pick up the changed metadata until it rescans for new/modified items.

The shows that you are having issues with... are they Sage Recordings, DVD sets, or shows "recorded" outside of SageTV.

I would enable logging and send me the log (pm or post it). There may be a bug.

[toricred]

They are Sage recordings. To save me the time of looking for it, where do I turn on the logging for this tool? Is it included in the Sage logging? (edit: I think I found it. Is it in the log4j.properties file? If so I've enabled it and will generate a log for you in a few minutes (mailcheck.log right?).

Since it is a Sage recording, does the metadata get scanned in at the "Automatic Library Rescan Frequency"? Is there a way to force it so I can make sure it is working?

[toricred]

Here's the log file.

[ehfortin]

Hi,

I've installed the most recent version of SageMC, SageTV and the metadatatool. It works pretty well for anything that is either a DVD movie or TV series. So, for example, it will add fanart for shows like House or Bones. It will also add fanart for my DVD folder for just about any movie that can be there. However, I'm recording movie from the TV as well and I don't seems to be able to get those fanart. Here is an example:

BEGIN MediaFile \\mce01\sagetv\10000BC-1604633-0.ts
Display Title: 10,000 B.C.

--- RAW Metadata Fields ---
MediaTitle:
MediaType:
SeasonNumber:

--- Calculated Metadata Fields ---
MediaTitle: 10,000 B.C.
MediaType: TV
SeasonNumber: 0

--- Fanart Paths ---
Background Path: \\MCE02\SAGETV\FanArt\TV\10,000 BC\Backgrounds
Banner Path: \\MCE02\SAGETV\FanArt\TV\10,000 BC\Banners
Poster Path: \\MCE02\SAGETV\FanArt\TV\10,000 BC\Posters

--- Fanart Single Files and Times ---
Background: null
Background Time (ms): 0
Banner: null
Banner Time (ms): 0
Poster: null
Poster Time (ms): 0
END MediaFile \\mce01\sagetv\10000BC-1604633-0.ts

This file is considered as "TV" as a MediaType. When I do a manual import, it is not showing any title. I've tried various TITLE= and I can see that if I put any movie name, it won't get a result. If I put any TV shows, it will suggest titles that could match.

So, it seems that if a MediaFile is tagged as MediaType, the fanart/metadata search will be only looked in a TV db repository where if it is a real movie (my DVD directory), it will get it from a movie db repository.

Is that the way it works? Is there a way to force the tool to try both type of repository? Or it should try both and I'm having some misconfiguration and/or bug?

BTW, I'm getting the same kind of result when trying to get the fanart from SageMC directly. TV shows are getting back with option and images where movie are not getting anything (movies that are recorded, not the one that were ripped from DVD).

At this point, I don't know what to do anymore. A little help would be appreciated.

Thank you.

ehfortin

[ehfortin]

I guess I have to publish the results of a last test I did and kind of do the job. Here is my command-line that successfully tracked most of the fanart for the recorded movies I have:

java -jar MetadataTool.jar --fanartFolder=\\share_name\sagetv\fanart --provider=imdb.xml \\share_name\sagetv

The interesting thing is that imdb.xml is one of the provider in my list in Sage so I would expect it to work there as well. I even moved it to the top of the list just to see if it would change something. Right now, it doesn't appear to change anything. So the manual command is working but a manual trigger in Sage is not getting any result.

I'm not formal on this as I didn't take time to do a proper test plan to see if it is in fact really acting differently but if I didn't make a mistake, that's the result I got at this point.

If we don't set anything, this utility has some default value (that can be see with --showProperties) but if the metadata.properties file exist, does the CLI utility use it? If not, is there a way to use it in order to make sure it is using the same parameter as SageTV?

I know the wiki is telling that we must configure the metadata.properties file for any permanent setting but I've not read anything about how the utility is actually finding the file by itself as I've not found a specific parameter to set it's path in command-line mode. For now, I assume that if it exist in the same directory I'm in while I execute the java -jar metadatatool.jar, it's using it. Is that correct?

Well, I guess I have to play more with this now that I Have at least one way of having it to work.

Any comment or information is welcomed.

ehfortin

[stuckless]

Originally Posted by toricred

They are Sage recordings. To save me the time of looking for it, where do I turn on the logging for this tool? Is it included in the Sage logging? (edit: I think I found it. Is it in the log4j.properties file? If so I've enabled it and will generate a log for you in a few minutes (mailcheck.log right?).

Since it is a Sage recording, does the metadata get scanned in at the "Automatic Library Rescan Frequency"? Is there a way to force it so I can make sure it is working?

I use nielms webserver and hit the "Rescan Media Library" link, or you can use the bmt command line and pass --refreshSageTV, or there is a way to tell Sage to refresh when you are in the videos menu (I never use it, so I don't know what it is off hand)

You log contained a single scan from the commandline, but I didn't see any indication that the automatic plugin was installed or being used.

[jaminben]

Originally Posted by stuckless

I use nielms webserver and hit the "Rescan Media Library" link, or you can use the bmt command line and pass --refreshSageTV, or there is a way to tell Sage to refresh when you are in the videos menu (I never use it, so I don't know what it is off hand)

Its here, see screenshot

[stuckless]

ehfortin, Unfortunately, that is the way that it was designed (not saying that it's correct). All Sage recordings are treated as TV. Each provider (imdb, tvdb, etc) is configured to allow a specific kind of search (ie, tv, movies, music). So imdb is configured to only accept "movies", which is why 10,000bc as a sage recording will never get passed to imdb.

The decision to treat all sage recordings as TV was used because I'm not sure how to tell if a sage recording is a tv show vs a movie. If there is a sage api call that can be querried to test if a sage recording is a movie, then post it here, and I'll see about update the code to use it.

[jaminben]

Originally Posted by stuckless

The decision to treat all sage recordings as TV was used because I'm not sure how to tell if a sage recording is a tv show vs a movie. If there is a sage api call that can be querried to test if a sage recording is a movie, then post it here, and I'll see about update the code to use it.

You could try:

IsTVFile(MediaFile)
IsVideoFile(MediaFile)
IsDVD(MediaFile)
IsMusic(MediaFile)
IsBluRay(MediaFile)

IsLibraryFile(MediaFile) Returns true if the specified MediaFile has been either imported using a library path or if this is a television recording that has had the 'Move to Library' operation performed on it.

Edit:

My bad, I misread the post. Ignore the above.

[jaminben]

I can filter TV recordings in menu's using the SageMC category "Movie". So something like:

GetProperty("sagemc/movie_category", "Movie")

would enable me to remove all Movies from a recorded TV list. I would be suprised if the default UI doesn't have something similar. This maybe of some help to you, I don't know

[ehfortin]

That's what I was to say. In "Recorded TV", we can filter by Series and by Movie. And based from what I'm seeing, it really differentiate between TV show and movies so it could probably be used to "guess" the appropriate mediatype.

As an alternative, can we cheat with provider? I've not really looked into creating a provider but is there a way to create a new one (that would actually be one of the already known like themoviedb.org or another) and linked it to a TV mediatype? If that can be done, we may be able to trick the system.

Thank you for your hard work. It is appreciated.


ehfortin

[razrsharpe]

Originally Posted by jaminben

I can filter TV recordings in menu's using the SageMC category "Movie". So something like:

GetProperty("sagemc/movie_category", "Movie")

would enable me to remove all Movies from a recorded TV list. I would be suprised if the default UI doesn't have something similar. This maybe of some help to you, I don't know

To complete this thought you need to query the recording category with GetShowCategory(MediaFile) and then compare against a list of possible categories to determine what is a movie, etc (using something like the code Ben posted above).

Personally I don't see much use for it because I never record movies with Sage... but that's just me

[Comitizer]

I think that if this is possible it is a pretty good idea. There are lots of times where it would be nice to have the fanart for a film that has been recorded.

I find that the plugin doesn't identify my TV shows that are imported into videos automatically. Is this because of the same sort of reason (it is using the "movies" scraper but the video is a TV show?).

[matt91]

I think this is BMT related...but possibly it might not be.

I have some videos that sage wasn't importing, despite me refreshing the imported media a few times.

I finally looked in the log, and it appears that the BMT plugin is getting stuck on a file and causing Sage to quit the rest of the imported media search.

log:

Sun 5/3 16:02:47.415 Starting to scan lib import root: G:\
Sun 5/3 16:02:47.450 Starting to scan lib import root: \\naslite-2\Disk-0\Videos
Sun 5/3 16:02:50.110 Starting to scan lib import root: C:\temp movie
Sun 5/3 16:02:50.760 Starting to scan lib import root: \\NASLITE-2\Disk-2\Videos
Sun 5/3 16:02:54.555 BMT: BatchMetadataTools 2.3.1; Handling File: \\NASLITE-2\Disk-2\Videos\Dora Dance Rescue\VIDEO_TS; arg: 
Sun 5/3 16:02:54.571 BMT: Reusing existing metadata for MediaFile: \\NASLITE-2\Disk-2\Videos\Dora Dance Rescue\VIDEO_TS
Sun 5/3 16:02:54.575 Seeker has finished the library import scan.

I'll try deleting and re-adding the imported media folder in question and see if that helps. However, this appears that it might be a bug.

thanks
matt


edit:
well, on a subsequent run, Sage has apparently continued on to find the remaining videos to import:

Sun 5/3 16:05:20.977 Starting to scan lib import root: G:\
Sun 5/3 16:05:21.011 Starting to scan lib import root: \\naslite-2\Disk-0\Videos
Sun 5/3 16:05:23.597 Starting to scan lib import root: C:\temp movie
Sun 5/3 16:05:24.244 Starting to scan lib import root: \\NASLITE-2\Disk-2\Videos
Sun 5/3 16:05:27.929 BMT: BatchMetadataTools 2.3.1; Handling File: \\NASLITE-2\Disk-2\Videos\Dora Dance Rescue\VIDEO_TS; arg: 
Sun 5/3 16:05:30.092 BMT: Could not automatically update: file:////NASLITE-2/Disk-2/Videos/Dora%20Dance%20Rescue/
Sun 5/3 16:05:30.096 BMT: Failed to Fetch Metadata for Media: \\NASLITE-2\Disk-2\Videos\Dora Dance Rescue\VIDEO_TS
Sun 5/3 16:05:30.134 Added:MediaFile[id=350034 A[350037,350035,"Dora Dance Rescue/VIDEO_TS",0@0325.22:39,0] host=house-server encodedBy= format=null \\NASLITE-2\Disk-2\Videos\Dora Dance Rescue\VIDEO_TS, Seg0[Wed 3/25 22:39:55.999-Wed 3/25 22:39:56.000]]
Sun 5/3 16:05:30.134 New Library DVD/BluRay MediaFile[id=350034 A[350037,350035,"Dora Dance Rescue/VIDEO_TS",0@0325.22:39,0] host=house-server encodedBy= format=null \\NASLITE-2\Disk-2\Videos\Dora Dance Rescue\VIDEO_TS, Seg0[Wed 3/25 22:39:55.999-Wed 3/25 22:39:56.000]]
Sun 5/3 16:05:30.342 BMT: BatchMetadataTools 2.3.1; Handling File: \\NASLITE-2\Disk-2\Videos\21 The Movie (2008)\VIDEO_TS; arg: 
Sun 5/3 16:05:30.360 BMT: Reusing existing metadata for MediaFile: \\NASLITE-2\Disk-2\Videos\21 The Movie (2008)\VIDEO_TS
Sun 5/3 16:05:30.407 Added:MediaFile[id=350038 A[350057,350039,"21 The Movie (2008)/21 ",0@0502.20:03,0] host=house-server encodedBy= format=null \\NASLITE-2\Disk-2\Videos\21 The Movie (2008)\VIDEO_TS, Seg0[Sat 5/2 20:03:04.999-Sat 5/2 20:03:05.000]]
Sun 5/3 16:05:30.407 New Library DVD/BluRay MediaFile[id=350038 A[350057,350039,"21 The Movie (2008)/21 ",0@0502.20:03,0] host=house-server encodedBy= format=null \\NASLITE-2\Disk-2\Videos\21 The Movie (2008)\VIDEO_TS, Seg0[Sat 5/2 20:03:04.999-Sat 5/2 20:03:05.000]]
Sun 5/3 16:05:30.981 BMT: BatchMetadataTools 2.3.1; Handling File: \\NASLITE-2\Disk-2\Videos\Indiana Jones and the Last Crusade\VIDEO_TS; arg: 
Sun 5/3 16:05:30.998 BMT: Reusing existing metadata for MediaFile: \\NASLITE-2\Disk-2\Videos\Indiana Jones and the Last Crusade\VIDEO_TS
Sun 5/3 16:05:31.066 Added:MediaFile[id=350058 A[350073,350059,"Indiana Jones and the Last Crusade/Indiana Jones and the Last Crusade",0@0429.19:59,0] host=house-server encodedBy= format=null \\NASLITE-2\Disk-2\Videos\Indiana Jones and the Last Crusade\VIDEO_TS, Seg0[Wed 4/29 19:59:52.999-Wed 4/29 19:59:53.000]]
Sun 5/3 16:05:31.066 New Library DVD/BluRay MediaFile[id=350058 A[350073,350059,"Indiana Jones and the Last Crusade/Indiana Jones and the Last Crusade",0@0429.19:59,0] host=house-server encodedBy= format=null \\NASLITE-2\Disk-2\Videos\Indiana Jones and the Last Crusade\VIDEO_TS, Seg0[Wed 4/29 19:59:52.999-Wed 4/29 19:59:53.000]]

I think that I had deleted the .properties file for this file on this run, but I'm not sure.

Anyway, it's fixed, but I'm still not sure why the DVDs weren't being imported before.

[toricred]

@Stuckless

I sent you a few PMs with the beginning pieces of the log file. Unfortunately I didn't catch it until I had almost 10MB.

I have more if you need it.