Utility: Batch Metadata Importer 3.x

[gplasky]

Originally Posted by sflamm

stuckless:

Did you see my idea/suggestion on allowing providers to have their capabilities demarcated and to use them in order to return all the appropriate information?

http://forums.sagetv.com/forums/show...&postcount=541

You mean like in item#43 from this post?

Configuration
http://forums.sagetv.com/forums/show...&postcount=576

Gerry

[sflamm]

No - I dont think that fits the bill for what I am saying. And is very confusing...

You dont want to have to make "composite providers".

All you want to do is:

1. Specify the name of a provider
2. Specify the capabilities of the provider (metadata, fanart)
This should probably have some granularity

- movieMetaDataProvider
- movieFanArtProvider
- tvMetaDataProvider
...

3. Put it in the provider list in order of use

The result would be a provider list like:

providerlist=dvdprofiler{movieMetaDataProvider}:moviedv{movieMetaDataProvider, movieFanArtProvider}

This is simple to specify and easy to understand the syntax and behavior.

You could easily see how the UI plug-in would act as well. You 'add' a provider and specify its capabilities. Then you order the providers in a graphical list.

[stuckless]

Originally Posted by sflamm

No - I dont think that fits the bill for what I am saying. And is very confusing...

You dont want to have to make "composite providers".

All you want to do is:

1. Specify the name of a provider
2. Specify the capabilities of the provider (metadata, fanart)
This should probably have some granularity

- movieMetaDataProvider
- movieFanArtProvider
- tvMetaDataProvider
...

3. Put it in the provider list in order of use

The result would be a provider list like:

providerlist=dvdprofiler{movieMetaDataProvider}:moviedv{movieMetaDataProvider, movieFanArtProvider}

This is simple to specify and easy to understand the syntax and behavior.

You could easily see how the UI plug-in would act as well. You 'add' a provider and specify its capabilities. Then you order the providers in a graphical list.

Good luck with that.... let me know when you have a patch ready... and I review for the project.

[babgvant]

Originally Posted by stuckless

Good luck with that.... let me know when you have a patch ready... and I review for the project.

[RedCoat999]

I cannot set the fanartfolder in the SageMC UI. It doesn't let me have control of that particular menu screen using the HD200. So i would like to use the CMD interface. How do i set "D:\Simons Files\Videos\Movies\"

It keeps saying "C:\Program Files\SageTV\SageTV\Files\Videos\Movies not found
Thanks

[stuckless]

Originally Posted by RedCoat999

I cannot set the fanartfolder in the SageMC UI. It doesn't let me have control of that particular menu screen using the HD200. So i would like to use the CMD interface. How do i set "D:\Simons Files\Videos\Movies\"

It keeps saying "C:\Program Files\SageTV\SageTV\Files\Videos\Movies not found
Thanks

From the commandline, you should be able to run the following command from the SageTV root folder

> java -jar MetadataTool.jar --fanartFolder=C:\YOUR_LOCATION "C:\Program Files\SageTV\SageTV\Files\Videos\Movies"

Note that if you have spaces in your paths, you'll need to "Quote" the "paths" or use the windows short folder name notation.

[razrsharpe]

Originally Posted by gplasky

http://forums.sagetv.com/forums/show...&postcount=541

You mean like in item#43 from this post?

Configuration
http://forums.sagetv.com/forums/show...&postcount=576

Gerry

thanks for pointing that out . That accomplishes what sflamm wants (albeit not as simply from the user's standpoint) and is exactly the functionality i was asking about earlier. that's what i get for not RTFM (aka this thread) more thoroughly.

[stuckless]

Actually there's a bug in the composite provider stuff right now, but I'm hoping to have new build out this week that should fix it, and few other issues.

[sflamm]

Just trying to provide input into the usability of the system (for all the end-users like me -- who do appreciate all your efforts). I thought you were interested in feedback.

the composite provider is pretty advanced for the common bear to setup. and it isnt accessible through the UI, is it?

I'm curious what you dont like about my proposal? Is there something specific?

[Comitizer]

OK, this might be a strange question and will probably expose the fact that I don't fully understand Sage but I think I'll ask it anyway... :-)

So, in my "videos" folder I have TV (looks like <show>/<season>/<episode>) but for some reason the BMI doesn't seem to be finding any meta data or fan art for the television shows. It seems to be working well for movies but there is nothing showing for TV.

It almost seems to me that the "Videos" section isn't make for television shows. Is this the case?

Also, the BMI worked great for the movies but for some reason the categories are all over the map. Is there any way to clean that up?

Thanks for the help. Great plugin!

[stuckless]

Originally Posted by Comitizer

So, in my "videos" folder I have TV (looks like <show>/<season>/<episode>) but for some reason the BMI doesn't seem to be finding any meta data or fan art for the television shows.

bmt has limited support (out of the box) for how it finds tv shows. The format that will work.... is something like,
<show>/<season>/<show> S<seasonE<episode>.avi

That is the out of the box functionality. You can configure something more specific to your setup, by adding a new filename scraper to the scrapers/xbmc/tvfilenames/ folder. (the filename scrapers use regex to pull out the series, season and episode for a given file path)

Also, the BMI worked great for the movies but for some reason the categories are all over the map. Is there any way to clean that up?

I think that I need to add some sort category/genre map so that we can have consistent genres. ie, if we see "Horror Film", then just use "Horror", etc.

[matt91]

Originally Posted by stuckless

I think that I need to add some sort category/genre map so that we can have consistent genres. ie, if we see "Horror Film", then just use "Horror", etc.

This would be very helpful. I was just thinking this weekend that I should go through the properties files themselves and start to clean up these genres so that they import properly (= into fewer categories). But, if I can just re-run the tool again in the future to get this metadata, than I may wait it out.

thanks
Matt

[stuckless]

Originally Posted by sflamm

Just trying to provide input into the usability of the system (for all the end-users like me -- who do appreciate all your efforts). I thought you were interested in feedback.

I've always said that I'm open to constructive feedback. Usually statement like, yours, don't fall into that category. Constructive feedback, usally takes a tone like.... "while your idea of a composite provider is flexible, perhaps a more granular approach, such as.... could be an alternative".... Your statement read like.... composite providers are simply the wrong way to do it... and here's the right way to do it....

the composite provider is pretty advanced for the common bear to setup. and it isnt accessible through the UI, is it?

Maybe I'm way to technical and far removed from the ordinary user, but I don't really see of eding a file and specifying the search and detail provider is complicated.

/compositeMetadataProviders/sample/name=Sample Composite Provider
/compositeMetadataProviders/sample/description=This is a sample Composite /compositeMetadataProviders/sample/searchProviderId=imdb
/compositeMetadataProviders/sample/detailProviderId=themoviedb.org
/compositeMetadataProviders/sample/compositeMode=2

probaby the composite mode, is the most complex part, but it's optional, and is only meant to describe how your want merge the details.

I'm curious what you dont like about my proposal? Is there something specific?

It's not about liking or disliking... I simply don't have a need for it. For the most part, a composite provider will do what you need, and personally I find it more readable than a single delimited line of configuration (as your proposed). But don't lose hope... this is an open source project... there was a time when I didn't have need to fanart as well... but someone else took it upon himself to add it...

I don't see the point in configuring dvdprofiler to NOT be a fanart provider, when it can in fact provide high quality images. I actually looked at one point about adding a specific fanart provider option, but it didn't make sense, since a provider details can already return images or metadata, and having the make a specific decision on how i was going treat the details from a given provider actually complicates the logic. So, in a composite provider of dvdprofiler and the moviedb.... the details (metadata and fanart) from both providers are actually merged together to provider a more complete set of metadata and fanart.

[jaminben]

Originally Posted by stuckless

Maybe I'm way to technical and far removed from the ordinary user, but I don't really see of eding a file and specifying the search and detail provider is complicated.

Sorry, I disagree. When you first look at it you get the "Oh my god, what am I supposed to do here" feeling. What this needs and I see has sort of been attempted in some form is a GUI. The biggest turn off for a user like me is if I can't work it out in 5mins then I don't want to know, I suspect that 95% of users would think the same thing.

I do however agree that if you sit down for 10mins and actually read the instructions it becomes quite simple, but this still doesn't detract from the fact that at first glance its a no-no.

Since I took the time to work your tool out, its now doing an awesome job of finding metadata and images for my movies, I don't need to lift a single digit anymore, well apart from the one to switch my system on

[stuckless]

Originally Posted by jaminben

Sorry, I disagree. When you first look at it you get the "Oh my god, what am I supposed to do here" feeling.

My point was that editing the file and specifying the configuration was not harder (and in my oppinion, easier) than simply providing a similar configuration on a single line. Both would require an understanding of the providers and syntaxes for configuration.

When I wrote, bmt, it had 1 goal... do what I needed it to do... but in doing that, I decided to make it flexible enough that it could what others needed it to do a well... If I had wrote it with no configuration options at all... then the responses would be.... "how come I can't simply tell it to use dvdprofiler instead of imdb"... but now that it's written with everything be configurable... i get, "it's too complicated... I just want a big red easy button"...

What this needs and I see has sort of been attempted in some form is a GUI. The biggest turn off for a user like me is if I can't work it out in 5mins then I don't want to know, I suspect that 95% of users would think the same thing.

I've asked for help in creating a gui... there are no takers.... so it's not going happen. I've written 3 guis for bmt, 1 in swing and 2 in web technologies. And I've even tried my hand at using the STV. The best hope for a gui is going to come from this community....because I sucked at every attempt to make gui

I do however agree that if you sit down for 10mins and actually read the instructions it becomes quite simple, but this still doesn't detract from the fact that at first glance its a no-no.

bmt is multi faceted.... you can, in it's simplest form, do nothing except run the command.

# java -jar MetadataTools.jar YOUR_FOLDER

or, read the documentation, and configure to your heart's content. The tool from the command line, documents it's command line and even it's configuration. --showProperties dumps every single configuration option with a description of what it's for. I don't think requiring someone to spend 10mins to review a tool is too much to ask. But if it is... then clearly this is wrong tool.... fortunately there are other choices.

[stuckless]

Originally Posted by matt91

This would be very helpful. I was just thinking this weekend that I should go through the properties files themselves and start to clean up these genres so that they import properly (= into fewer categories). But, if I can just re-run the tool again in the future to get this metadata, than I may wait it out.

thanks
Matt

What i'm thinking is that we could have a Genre lookup table (file) where we could list the Genre and a regex of possible matches...
ie,
Horror: Horror, Scary, Fright

When the properties are processed, i can see if the current genre contains, "Horror", "Scary", "Fright", and if so, then assign it to the "Horror" genre.

If we go through the list and we haven't found a genre, then we can either add the genre text as is... or assign it to a cusom "Unknown" genre.

That's my take on the process.

[sflamm]

1st off - let me say as a software professional myself - I appreciate the huge amount of effort that goes into writing the BMT and the great job you are doing.

2nd - let me also say, as a previous user of Windows MCE (for many many years, it is BECAUSE of the features like the ones you have added to SageTV I have decided to switch. That's how much they improve the overall experience

Please take my input, as input. I'll try to be a little more careful in how I phrase my thoughts.

Here are some thoughts to consider on BMT (having read all the docs, all the threads, and having run both the plug-in and the command line tools, and I take the time to try to contribute to the thread/discussion):

1. For a standard user, they would prefer to install the plug-in and let it handle the meta-data/fanart automatically - updating meta data/fanart periodically as source media is added (without manual intervention).

2. Any changes to behavior would realistically need to be handled through the UI.

3. Editing files manually and configuring setttings without a GUI is highly unlikely for the standard user - who frankly would be overwhelmed and concerned that by doing the wrong thing they would break their installation

4. More documentation would be VERY helpful. Currently the doc is sparse - and really not written for the layman or even for the more advanced user.

You have done a great job of creating all sorts of options. But unfortunately that also creates confusion - even for software types. My experience is that the only way to really figure out how it works not is with trial and error from the command line.

It would be great if someone could write a word doc that had example scenarios and the step-by-step on how to set them up. I would volunteer to help, but to be frank, I am not 100% sure on the setup myself - as things I have tried that I believe will ultimately work when you work out all issues, may not be working currently. And like all good projects, the design/features are fluid to allow for new improvements etc.

If you are willing to provide me with answers to questions, I am willing(volunteering) to write/re-write the doc.

Some thoughts on previous thread:

1. I wasnt recommending that there be necessarily a single line describing the provider use (that was just to explain my thoughts). To that end - I also wasnt suggesting that dvdprofiler isnt a good source of fanart (I bought it for the high quality images) - that too was just for purpose of writing an example.

2. I think the Genre mapping via lookup is the right flexible approach. It ultimately will need a UI as well Sorry to say it, but.... think about the common bears...

[jaminben]

Originally Posted by stuckless

Both would require an understanding of the providers and syntaxes for configuration.

Thats what I mean, its too hard for the average user. For example, running an ms-dos prompt and using java -jar MetadataTool.jar to get a list of properties took me half an hour to work out. Mainly because I haven't had the need to do this on a regular basis.

Originally Posted by stuckless

When I wrote, bmt, it had 1 goal... do what I needed it to do... but in doing that, I decided to make it flexible enough that it could what others needed it to do a well... If I had wrote it with no configuration options at all... then the responses would be.... "how come I can't simply tell it to use dvdprofiler instead of imdb"... but now that it's written with everything be configurable... i get, "it's too complicated... I just want a big red easy button"...

bmt is multi faceted.... you can, in it's simplest form, do nothing except run the command.

# java -jar MetadataTools.jar YOUR_FOLDER

or, read the documentation, and configure to your heart's content. The tool from the command line, documents it's command line and even it's configuration. --showProperties dumps every single configuration option with a description of what it's for. I don't think requiring someone to spend 10mins to review a tool is too much to ask. But if it is... then clearly this is wrong tool.... fortunately there are other choices.

Agreed, You can't please all of the people some of the time, but you can please some of the people all of the time I sense your getting a little frustrated and that wasn't my intention. All I'm saying is, in its current form its not user friendly. I will learn to use it because I want to, there are others that will only bitch about it being too complicated and not bother learning.

Originally Posted by stuckless

I've asked for help in creating a gui... there are no takers.... so it's not going happen. I've written 3 guis for bmt, 1 in swing and 2 in web technologies. And I've even tried my hand at using the STV. The best hope for a gui is going to come from this community....because I sucked at every attempt to make gui

I'll have a go at making a GUI for SageMC, it'll be a long and painfull process but in my opinion is the best method for changing settings once the tool has been installed. As you can see from my above comments I'm not the most talented programmer or even a slightly compentent one but I'll do my best and learn what I need to.
I realise that your busy with other projects and may not want to spend the time working on this with me, if this is the case then I'll gladly carry on using your tool no matter how hard it is to use as I want to use it and will take the time to do so.

[sflamm]

jaminben: are you running the tool via command line to update your metadata/fanart periodically? Or do you have a method to do it automatically now? I tried the plug-in for that, but I think it still has a few bugs preventing it from doing that...

[jaminben]

Originally Posted by sflamm

jaminben: are you running the tool via command line to update your metadata/fanart periodically? Or do you have a method to do it automatically now? I tried the plug-in for that, but I think it still has a few bugs preventing it from doing that...

I use the MetadataFanartTools-SageMC.xml import, setup my metadata.properties to how I wanted (image sizes etc) and set Sage to scan for files automatically. Then from within the SageMC UI tool I added and deleted the providers I needed. All I do now is copy the movie over to my server and within 5 mins it has all the metadata and fanart for that film. I've not had any issues with it grabbing the required data.