SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations > Batch Metadata Tools
Forum Rules FAQs Community Downloads Today's Posts Search

Notices

Batch Metadata Tools This forums is for discussing the user-created Batch Metadata Tools for SageTV.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 12-05-2010, 10:04 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
Can you do this in BMT?

I apologize since this is probably a stupid question, but here goes...

I have a bunch of Kids Shows that are in my Video Library organized under folders and then (generally) with file names that come from episode names. The folder names are approximately, but not necessarilly exactly, the name of the show.

For example my folder structure is like this...

\Kids Shows\Dora
\Kids Shows\Diego
\Kids Shows\Backyardigans

FYI - the technically correct name for the shows is Dora the Explorer and Go Diego Go. When I browse these files in BMT it thinks that these are movies. Occasionally where the episode name matches a movie (ie. Lost and Found) BMT (at least I think it is BMT) will download Fanart for that movie.

I can go through these files one by one and Search for Metadata and save but this is a slow and painful process. Is there a way to do this in Batch Mode in BMT - ie. tell Sage that the title is Dora The Explorer for all of the shows in this folder and then have it go to town and try to match the episode name with the file name to download metadata and attach the correct file name?

Sorry if this is somewhat unclear.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #2  
Old 12-05-2010, 10:21 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
I apologize since this is probably a stupid question, but here goes...

I have a bunch of Kids Shows that are in my Video Library organized under folders and then (generally) with file names that come from episode names. The folder names are approximately, but not necessarilly exactly, the name of the show.

For example my folder structure is like this...

\Kids Shows\Dora
\Kids Shows\Diego
\Kids Shows\Backyardigans

FYI - the technically correct name for the shows is Dora the Explorer and Go Diego Go. When I browse these files in BMT it thinks that these are movies. Occasionally where the episode name matches a movie (ie. Lost and Found) BMT (at least I think it is BMT) will download Fanart for that movie.

I can go through these files one by one and Search for Metadata and save but this is a slow and painful process. Is there a way to do this in Batch Mode in BMT - ie. tell Sage that the title is Dora The Explorer for all of the shows in this folder and then have it go to town and try to match the episode name with the file name to download metadata and attach the correct file name?

Sorry if this is somewhat unclear.
You can do what you need to do using the MediaTitles.xml, which is an xml that helps bmt select better results. You'd need to add an entry, something like...
Code:
	<match>
		<regex>[\\/]Dora[\\/]</regex>
		<title>Dora The Explorer</title>
	</match>
or

Code:
	<match>
		<regex>[\\/]Dora[\\/]</regex>
		<metadata type="tv" name="tvdb">74697</metadata>
	</match>
Once you make the the change, you can then reload the MediaTitles.xml by restarting the sagetv server.

The MediaTitles.xml is located in STVs/Phoenix/scrapers/
Reply With Quote
  #3  
Old 12-06-2010, 12:56 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
I see that 74697 is the show ID in tvdb for Dora the Exploer. Will this code then cause BMT to reclassify the files in the folder as TV rather than Movies? I don't want them to show up in Recorded TV, I want them to stay in the Video Library - do I have to do anything else to ensure that this is the case?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #4  
Old 12-06-2010, 01:19 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
I see that 74697 is the show ID in tvdb for Dora the Exploer. Will this code then cause BMT to reclassify the files in the folder as TV rather than Movies? I don't want them to show up in Recorded TV, I want them to stay in the Video Library - do I have to do anything else to ensure that this is the case?
By default, when BMT adds metadata and fanart to TV shows (ie non recordings), they remain as non recordings in your video libary, but they are "tv" files as far as bmt and phoenix is concerned. BMT will only import them into the recordings, if you explicity tell it to in the bmt configuration. By default "Import TV Shows as Recordings" is not enabled.
Reply With Quote
  #5  
Old 12-06-2010, 01:41 PM
bialio's Avatar
bialio bialio is offline
SageTVaholic
 
Join Date: May 2007
Location: Frisco, TX
Posts: 3,445
<whacked>

<EDIT> I take it all back - now it's coming clearer to me. The regex above looks for "Dora" in the name of the fully qualified file name (path and all) surrounded by either a / or \ - so it should work.

btl.
__________________
PHOENIX 3 is here!
Server : Linux V9, Clients : Win10 and Nvidia Shield Android Miniclient

Last edited by bialio; 12-06-2010 at 01:45 PM.
Reply With Quote
  #6  
Old 12-23-2010, 09:26 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
@stuckless - I tried editing the mediatitles.xml file as you suggest in post 2 but it doesn't seem to be working at all. Is there any way to troubleshoot via logs, etc?

Here is my current MediaTitles.xml file. Note that the relevant files are in \\server\videos\isabella's shows\dora and these files are being imported into Sage - I can see them when I browse in the BMT web app.

Code:
- <titles>
- <!-- 	A Match can contain a regex that will be used to match against the path.  If it matches, then it will
	then set the fields, title, year, or id, into the search query.
	
  --> 
- <match>
  <regex>[\\/]Babylon\s*5[\\/]</regex> 
  <title>Babylon 5</title> 
  <year>1993</year> 
  <metadata type="tv" name="tvdb">7072</metadata> 
  </match>
- <match>
  <file>/home/movies/shared/FindingNemo.avi</file> 
  <title>Finding Nemo</title> 
  <year>2003</year> 
  <metadata name="imdb">tt0266543</metadata> 
  </match>
- <match>
  <regex>[\\/]Dora[\\/]</regex> 
  <metadata type="tv" name="tvdb">74697</metadata> 
  </match>
  </titles>
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #7  
Old 12-23-2010, 09:49 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
When you edit the mediatitles.xml, they have to re-loaded, before they will take effect. You can reload them using the Refresh MediaTitles in Configure -> Refresh Configurations.

Also, I'm not sure, but it may be case sensitive. So, if you directory is "dora", then you'll need to specify a lowercase 'd'.

So, is it not finding dora, or is it finding dora, but then not finding a match on the episode for dora?
Reply With Quote
  #8  
Old 12-23-2010, 10:22 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
I restarted the Sage server and no additional metadata has been found for the files. The case appears to be consistent, as in the D is capitalized in both the folder structure and in the mediatitles file.

Can I manually run BMT against a file to see if it works? If so, how? I have just been assuming that the automatic nature of BMT will find the metadata.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #9  
Old 12-24-2010, 07:45 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
I restarted the Sage server and no additional metadata has been found for the files. The case appears to be consistent, as in the D is capitalized in both the folder structure and in the mediatitles file.

Can I manually run BMT against a file to see if it works? If so, how? I have just been assuming that the automatic nature of BMT will find the metadata.
From the web ui, you can navigate to the file in question, and then click "Find Metadata".

Could you post up a couple of dora filenames that you are having issue with, and your complete mediatitles.xml file.
Reply With Quote
  #10  
Old 12-24-2010, 09:47 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
From the web UI I can get it working fine as I change type from movie to TV make sure the title is Dora the Explorer and put in the file name as the episode - I hit search and it generally finds the episode But this process takes a lot of manual intervention for every file. Note that my filenames are the valid episode names as I used SJQ to transocde files from MPEG to AVI and put the resulting file in the Dora folder with the episode name as file name. If I had to I could then use SJQ to run BMT in a batch file if that was possible.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #11  
Old 12-24-2010, 10:50 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
From the web UI I can get it working fine as I change type from movie to TV make sure the title is Dora the Explorer and put in the file name as the episode - I hit search and it generally finds the episode But this process takes a lot of manual intervention for every file. Note that my filenames are the valid episode names as I used SJQ to transocde files from MPEG to AVI and put the resulting file in the Dora folder with the episode name as file name. If I had to I could then use SJQ to run BMT in a batch file if that was possible.
From the webui, when you click find metadata, then Discover Defaults, does it appear to fill in the correct information. The 'Discover Defaults' button uses the same process as the automatic scanner for filling in the query.

If you email me the log files, when something fails, then I can check to see what bmt is using for the data, and see why it's failing. This should be working for something as simple as Dora.
Reply With Quote
  #12  
Old 12-24-2010, 12:32 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
Here is an example for one show. The full path is \\server\videos\Isabella's Shows\Dora\Berry Hunt.avi

When I go into the video via browse it says Media Type: Movie, Movie Title: Berry Hunt. I click on Find Metadata and then Discover Defaults. All that causes it to do is to change the Search Type from Movie to TV and the Source to thetvdb.com. It keeps the Search Title as Berry Hunt.

If I manually change the Search Type from Movie to TV then the dialog box expands to show many more options, but it keeps the search title at Berry Hunt and it doesn't find anything. If I change Search Title to Dora the Explorer and Episode title to Berry Hunt and click Search then it finds Dora the Explorer. I click on that and it imports valid metadata.

What logs should I be emailing to you and do I have to enable logging (if I have not already done so) and if so, where?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #13  
Old 12-24-2010, 01:39 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
Here is an example for one show. The full path is \\server\videos\Isabella's Shows\Dora\Berry Hunt.avi

When I go into the video via browse it says Media Type: Movie, Movie Title: Berry Hunt. I click on Find Metadata and then Discover Defaults. All that causes it to do is to change the Search Type from Movie to TV and the Source to thetvdb.com. It keeps the Search Title as Berry Hunt.

If I manually change the Search Type from Movie to TV then the dialog box expands to show many more options, but it keeps the search title at Berry Hunt and it doesn't find anything. If I change Search Title to Dora the Explorer and Episode title to Berry Hunt and click Search then it finds Dora the Explorer. I click on that and it imports valid metadata.

What logs should I be emailing to you and do I have to enable logging (if I have not already done so) and if so, where?
OK, I see the issue.

The existing set of scrapers parse all information from the filename. ie,
Dora - Hunt.avi would work, but because your series name is in a folder called Dora, then the TV scrapers do not work

At this time, there is nothing BMT can do, unless we create a new TV scraper that can work with your media. I'll look at doing that, but I wouldn't include this scraper with the regular build, but I can provide one, and then you can manually add it.
Reply With Quote
  #14  
Old 12-24-2010, 02:00 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
That's fine - I can easily rename my files with a batch file renamer and then they should work. The reason that I set up the files this way is that my daughter will also just play the files by clicking on the file in the thumbnail view of Windows explorer - I have a couple of touchscreen monitors that facilitate this. But I guess adding a prefix of "Dora - " won't really change anything and the ordering of files will still be the same as the alphabetic order won't kick in until after the Dora.

But I would encourage you to think about adding this functionality to the basic release as it makes sense to organize shows based on folders - but then it becomes redundant to also include the show name in the filename. At least that is my opinion and I am sticking to it!

The other thing that could work is if you could select a bunch of files and then set the Show Title proprety for all of them files at once. You can kind of do this with MP3 tags in iTunes, not that iTunes is an app worth emulating. I often use this functionality in iTunes to make naming of bands consistent rather than having "Beatles", "The Beatles" and "Beatles, The" showing up as three different artists.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #15  
Old 12-24-2010, 02:51 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
That's fine - I can easily rename my files with a batch file renamer and then they should work. The reason that I set up the files this way is that my daughter will also just play the files by clicking on the file in the thumbnail view of Windows explorer - I have a couple of touchscreen monitors that facilitate this. But I guess adding a prefix of "Dora - " won't really change anything and the ordering of files will still be the same as the alphabetic order won't kick in until after the Dora.

But I would encourage you to think about adding this functionality to the basic release as it makes sense to organize shows based on folders - but then it becomes redundant to also include the show name in the filename. At least that is my opinion and I am sticking to it!

The other thing that could work is if you could select a bunch of files and then set the Show Title proprety for all of them files at once. You can kind of do this with MP3 tags in iTunes, not that iTunes is an app worth emulating. I often use this functionality in iTunes to make naming of bands consistent rather than having "Beatles", "The Beatles" and "Beatles, The" showing up as three different artists.
Actually, since you are using the Series - Episode name, then you'll want to use two dashes.

Dora -- Hunt.avi

It's hard for a TV scraper to know it's a TV file without some sort of indicator, so double dash is used to say, "hey, this is tv show and espisode title".
Reply With Quote
  #16  
Old 01-15-2011, 06:12 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
@stuckless

I have renamed all of the files in my folder to start with "Dora--" and I have also renamed the folder to "Dora the Explorer" from "Dora" (this will facilitate automatically moving files to this folder).

When I go into BMT and manually select one of the shows and click on Discover Defaults it finds all of the metadata. I have manually updated 5 shows.

But BMT doesn't seem to want to automatically go through and update the metadata for all of these files.

I have refreshed the library a couple of times and restarted Sage server. I have also tried Batch Operations - Associate TV Series Information. I have the right entry in the MediaTitles.xml file which is obvious since the Discover Defaults works perfectly.

Do I have to do something else to have BMT rescan these files and generate the info?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #17  
Old 01-16-2011, 08:48 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
@stuckless

I have renamed all of the files in my folder to start with "Dora--" and I have also renamed the folder to "Dora the Explorer" from "Dora" (this will facilitate automatically moving files to this folder).

When I go into BMT and manually select one of the shows and click on Discover Defaults it finds all of the metadata. I have manually updated 5 shows.

But BMT doesn't seem to want to automatically go through and update the metadata for all of these files.

I have refreshed the library a couple of times and restarted Sage server. I have also tried Batch Operations - Associate TV Series Information. I have the right entry in the MediaTitles.xml file which is obvious since the Discover Defaults works perfectly.

Do I have to do something else to have BMT rescan these files and generate the info?
BMT only automatically scans new files added to the collection, not existing files. So to rescan files that already exist... You use the web ui, navigate to the the dora folder, and then click the "Update Metadata" button and then uncheck "Only update items that have not been updated previously". That will perform a scan from the web ui, updating all items in the current folder (and sub folders).
Reply With Quote
  #18  
Old 01-16-2011, 09:47 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
I tried that but I don't think it worked as it came back almost immediately with: Scan (Dora the Explorer) Complete ( 0 : 124). I take it that means that 0 of 124 files had metadata updated?

edit - there is definitely something wrong as when I scan other folders you can see the files being scanned - but not in the Dora folder and in some other folders. Are there any logs that I should check and does logging have to be turned on first?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server

Last edited by wayner; 01-16-2011 at 09:53 AM.
Reply With Quote
  #19  
Old 01-16-2011, 11:09 AM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,713
Quote:
Originally Posted by wayner View Post
I tried that but I don't think it worked as it came back almost immediately with: Scan (Dora the Explorer) Complete ( 0 : 124). I take it that means that 0 of 124 files had metadata updated?

edit - there is definitely something wrong as when I scan other folders you can see the files being scanned - but not in the Dora folder and in some other folders. Are there any logs that I should check and does logging have to be turned on first?
From the web ui, under the Help, you can create support request. That will create a zip file of your log files. You can send me that, PLUS, include your mediatitles.xml and I can see what's going on. I may not be able to look at it today, but I try to look at it tomorrow.
Reply With Quote
  #20  
Old 01-16-2011, 11:22 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,491
I renamed the files from "Dora -- Boo!.avi" to "Dora the Explorer -- Boo!.avi" and that seemed to do the trick. The problem may have been that there was another TV show called I Married Dora that comes up. Now I hope to truncate the "Dora the Explorer -- " from the filenames
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
BMT issues heffneil Batch Metadata Tools 4 12-15-2010 05:50 PM
What is BMT? Pelicanjoe Batch Metadata Tools 45 12-09-2010 08:58 AM
Archived and BMT kmp14 Batch Metadata Tools 2 08-20-2010 12:51 PM
BMT getting wayner Batch Metadata Tools 6 08-13-2010 06:33 AM
BMT: Read this if you don't see any BMT topics Opus4 SageTV Customizations 0 04-25-2010 09:17 PM


All times are GMT -6. The time now is 03:58 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.