|
SageTV Customizations This forums is for discussing and sharing user-created modifications for the SageTV application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss customizations for SageTV version 6 and earlier, or for the SageTV3 UI. |
|
Thread Tools | Search this Thread | Display Modes |
#61
|
|||
|
|||
Quote:
Quote:
Quote:
Anyway here's some of the issues I found. I've made copies of the directories and zero size copies of the files and put them in the attached zip so you can run local tests if you wish. 1)I found a file that will crash the scraper. It's true this file is in a format that really shouldn't work with your scraper(at least I'd be surprised if you got it too return the correct result, with this little info on the show, or movie in this case). Still it will cause the scraper to stop(crash at this file) processing a group of subdirs if this type of file is in one of the subdirs. I only found the trouble-maker(the file listed in the code window below) quickly because I was running my batch and could see where it stopped and that only left 2 files that could have caused it. Here's a copy of the error text below(the whole videolist log, batch text log and zero size copy of the file are included in the zip in the directory called "Ghost in the Shell movies") Code:
E:\Documents and Settings\Michael\Desktop\Ghost in the Shell Movies\Ghost In The Shell.ogm Loading episodes... Traceback (most recent call last): File "videometadata.py", line 74, in <module> IndexError: list index out of range 3) This isn't truly a bug. Since you told me this would happen in a earlier post. Still I remember your saying you'd look into trying to handling it, so here's an example. It's for the show Inuyasha. The are many(all 167 eps) example files in the zip in the dir called "Inuyasha". It's the issue you mention on a 3 digit episode # being parsed as 1st digit season and last digit episode #. Of course this results in .my files created, but they are all mismatched after you get into the higher digits. I really only tested the last season(160-167) extensively. Still it produced the behavior you said would likely happen. Have/will you be able to work this in? I'd be totally fine with a switch I had to add(or set via CLI shortcut/batch) if I was processing a dir full of mutliseason/3 digit episode # shows. Such a switch would tell the app to use the showid.txt(if even needed, your app is very smart at finding the rigt show) and just use the 3 digit episode #s to make the .my files and forget seasons, like the list here created by listing all seasons(aka season=0). I understand this isn't the format you made your app for but I have so many shows in this format(many already burnt off to DVD) that to have this functionality is very important, at least to me. So thanks for at least considering it. One other thing about the Inuyasha dir. It does contain several episodes where 2-parters have been merged into 1 file. I don't want you to think I'm somehow asking you to handle these few oddball episode filenames. I realize they're format is too weird.ie "Inuyasha - 147+148 - The Tragic Love Song of Destiny" I can easily make my own merged .my file for these few anomalies. It's the manual(only effective way without your app) creation and renaming of hundreds of .my files for these 3 digit episodes that is my real concern. Last edited by justme; 07-24-2007 at 08:33 AM. |
#62
|
|||
|
|||
First sorry for the 2 posts in a row but It suddenly occured to me that i didn't include a log for a test run on Inuyasha. I just described it and fogot to add the tests to the zip. I just reran a sample(small group of files) test and zipped the logs,zero size files, and the mostly(looks like every episode's .my file above 009 is wrong, but that fits the behavior you said to expect) mismatched .my files.
I thought that maybe adding Season 0(didn't try that before, but my last post made me curious to try it out) to the directory title would allow the app to go to just a episode digit parsing logic mode. Some other thoughts, this behavior makes me very curious how the CLI will handle shows with #s in the Episode Title of the filename. Even if the suggested format of S??xE?? is used. ie How would "Inuyasha-S01xE06-The 7 vilians" be parsed. Would the CLI just ignore the 7 since it's already filled it's slots for season and episode? If so does that mean the first # starts the logic chain? If so what about shows that have #s in their names like "Eureka 7" or "Samurai 7"(7 seems to be a populiar # with anime show tittles )? If the CLI tries to start it's chain logic on the first(or more depending on title ie "4400") #(in this case the "7" that's part of a Show Title), could the CLI be sanity checked using the showid.txt to allow it to skip the Show Title #s("7" in this example) and then pick up where it normally would for a proper parse? Just thinking out loud here about possible issues, and didn't know if you'd considered/tested them. It seems like The CLI is starting at the first digit and ignores the rest after it fills it's slots, based of this small Inuyasha test. This seems the safer/simpler way to go but does make me wonder about the issue of #s in the Show Title. ie Would "4400-05-The 11 heroes" be parsed correctly parsed with a proper showid.txt? BTW:All the episode titles above where made up by me as examples of possible(may all work great) problems, they don't actually exist in those Shows, at least to my knowledge. Anyway I'm off to get some sleep. If there's no post with some feedback(I wouldn't be suprised if roxy99 already thought of all this and had it worked out) I'll run some tests with some Shows/Episodes that are like this to see what happens. /Peace |
#63
|
|||||
|
|||||
Quote:
Quote:
Here are some of the major new features: -Name based fallback detection: If the program fails to identify a show based on Season and/or Episode#, then the program will try to match the show with the episode title. This will alse allow you to have strictly name based files such as your file E:\Documents and Settings\Michael\Desktop\Ghost in the Shell Movies\Ghost In The Shell.ogm As you can appreciate, the present version is not taking advantage of name based files. The app looks for a season and episode # but fails. This will be taken care of now. -You'll be able to use -t -s settings to override the automatic detection engine. -Episode based numbering schemes will work now. Presently, as soon the episode # is higher than 99, eg 106- the app will assume Season 1 Episode 6 which is wrong. You must however name your file in a way to tell the application that you're only using episode # and no season #. You do this by preceding Ep.# with 'EP': IE- EP106: Do not use simply 106. Quote:
Its simpler than you think. The app uses a filter to exclude non-aired episodes. If you check that show, TV.com lists the episodes with Air date 0/0/0. The filter causes the app to ignore them. This filter was necessary to remove some of the trash on TV.com, (ie, special episodes that throw off the count, unaired filler episodes, etc.). With the new name based fall back scheme, I'll remove that filter so that unaired episodes are included in the episode lists scraped from TV.com. So this filter will no longer be needed. Quote:
Quote:
Just don't do anything weird with the episode title string or else I can't really help you.
__________________
http://www.hydranterouge.com |
#64
|
||||
|
||||
So the name recognition is used as a fallback or in conjunction with the previous detection methods?
IE: will it only come into play if it can not find the Season and Episode? Sometimes, some people might have there episodes named wrong or TV.com will have the airing mixed up. Will it find say 3x12-Blah_Blah_Blah.mp4 even though Blah_Blah_Blah is really 3x13? Or will it also check the title and create 3x12-Blah_Blah_Blah.my but have the information from the actually 3x13 episode in it?
__________________
If this doesn't work right, Then: "I'm going to blow up the Earth!" |
#65
|
|||
|
|||
Quote:
PS- Can you post up a .properties file that I can work with? PPS- Just in case someone is reading for the first time. This is only proposed for the new coming version.
__________________
http://www.hydranterouge.com Last edited by roxy99; 07-24-2007 at 11:17 AM. |
#66
|
|||
|
|||
Roxy...sorrry to bump my questions up, but you may have missed them at the botton of page 3....
Thanks for any help... Gary Ellis |
#67
|
|||
|
|||
Quote:
If the only difference is a USB drive then I'm curious: what in the full path name is preventing show id detection? Please post up your videolog.txt file. Make sure also that the title part of the path is not burried in the pathname in any illogical place. The most reliable organization is: <drive:>\misc path\Title path\Season\filename.ext OR <drive:>\misc path\Title path\filename.ext Drive can be any windows drive map letter. Misc path can be any number of layers deep. From there on, you need to be logical in your storage.
__________________
http://www.hydranterouge.com Last edited by roxy99; 07-24-2007 at 12:06 PM. |
#68
|
|||
|
|||
Here it is.... The C: drive is a USB drive....and the path is correct..
Like I said, when I put this on my laptop it works fine. Thanks, Gary Ellis |
#69
|
||||
|
||||
Here is a properties file created from Jerekiwi's RW Metadata stvi...
Just rename it from .txt to .properties
__________________
If this doesn't work right, Then: "I'm going to blow up the Earth!" |
#70
|
|||
|
|||
Is it the same USB drive that you move to your labtop and it works on the laptop? How does the USB drive show up in windows explorer of the HTPC?
__________________
http://www.hydranterouge.com |
#71
|
|||
|
|||
Thanks does mp4 also appear in the filename? ordoes the mp4 extension get dropped and replaced with .properties?
__________________
http://www.hydranterouge.com |
#72
|
|||
|
|||
This is for Justme only. Test on a backup folder first!
I'M USING THE .RAR COMPRESSION FORMAT in these attachements. I cheated and simply renamed the .rar to .zip. You'll have re-rename the .zip to .rar and then decompress this multipart attachement. This will rename a dragged/dropped folder with names to include the prefix EP in the episode # EG: Title - ### - Episode name.ext Becomes: Title - EP### - Episode name.ext Very important that the filename have the - dividing the sections.
__________________
http://www.hydranterouge.com Last edited by roxy99; 07-24-2007 at 02:24 PM. |
#73
|
||||
|
||||
Quote:
ArmyWives-OnlytheLonely.mp4.properties I renamed it: ArmyWives-OnlytheLonely.mp4.txt so that it could be uploaded...
__________________
If this doesn't work right, Then: "I'm going to blow up the Earth!" |
#74
|
|||
|
|||
The USB drive looks like a normal drive on my computer....
I did not attach the drive to my laptop...I copied the directory and video file to my laptop hard drive. And it works fine... I'm going to try it on my normal hard drive on my home theater computer and see if I get the same error.. But, when I look at it..the applicaton is certainly getting the right title, its just not finding it on line....That's why I wondered if the security may be a problem. Thanks, Gary Ellis |
#75
|
|||
|
|||
Quote:
If you can reproduce the error on the usb drive on another non-htpc pc, then could be the usb drive. However on your laptop the same usb drive worked hence I agree that it maybe has nothing to do with the usb drive but firewall or something preventing the app from fetching the url data.
__________________
http://www.hydranterouge.com |
#76
|
|||
|
|||
Quote:
I did attach a zip with 4 videolist log files. Their names pretty much say what they are. I'm trying(not too good at it) to keep this post shorter. They really won't(at least I don't think) tell you anything about your scraper that you didn't already know. Besides given your excellent explanation of the scraper's logic, in post #63, I think all the behavior in these logs is exactly as it should be for the current(1.1.1.6) version. It definitely sounds like you know where all the issues I described(or posted about earlier) are coming from and have a fix in the works. One Note, about the logs, in case you do decide to check them:I let the 1st run go all the way thru, but I interrupted(escaped the batch) the last 3 runs after they had processed a good sample of files. Letting it run all 4 test on 163 files 4 times would have taken quite a while, so I cheated for the last 3 tests. The features and functions you described for the upcoming version/versions sound excellent. I look forward to trying it out. Especially the 3 digit episode support. I'll make sure that all my shows(on my hard drive at least) get "EP" added(thanks to your app) to their filename if they go over 2 digits for their episode #. Quote:
BTW: Does anybody here know of a freeware alternative to an app called FakeTwin? This relates to me handling(using the scraper) files already burnt off to DVD. Many in a bad(3 digit episode number with no "EP") format. Since I can't write the .my files to a DVD(no write access) or even change the filenames(the scraper can't help here since even it lacks write access) of DVD files(to include "EP") if they're in a "bad" format. I need an app like this so I can change the filenames on my hard drive to a format that can be properly parsed. I can then remove the "EP", or whatever from the .my file's names and have them be named the same as the video files on the DVD. Once I move the .my files into my "global .my directory" they will then work with their matching DVDs using JREkiwi's metadata import. Key features: Copy dir structure(including subdirs), and all files in them, to a new/specified location. The files created in the "copy" of the directory should be 0 byte size. This results in almost no space used for a large tree structure. Such an app could also be usefull for testing the scraper on different machines/user setups. We'd be able to share/send exact copies of parts(or all) of our library tree to others to test. All this without actually sending any real videos. Just exact copies of the filenames since the files are empty. Oh well, so much for making a shorter post. /peace Last edited by justme; 07-25-2007 at 04:58 AM. |
#77
|
|||
|
|||
No joy here...
I copied the video files to a directory on my internal Hard drive and I turned off security on the system...same exact problem...For some reason the title is not found. And I know its a good name because it works on my laptop... Anybody...? any ideas.? Another questions... what is a .properties file? and where do I put it? Thanks, Gary |
#78
|
|||
|
|||
Quote:
Just make sure that you use the exact path on the laptop as on the HTPC and run the same test directory in both places. Have you also tried the obvious like rebooting your htpc? Restaring your router? Testing the app with no other PC's running on the network?
__________________
http://www.hydranterouge.com |
#79
|
||||
|
||||
Quote:
See this post for info on .properties files. Last edited by Armondole; 07-25-2007 at 08:32 AM. |
#80
|
|||
|
|||
garyellis, I thought I might try to help as well since everyone has been so kind to me.
One other thing you might try, if you haven't already... If your network is setup to allow shared drives(and one of them is that working laptop drive), try using drag n' drop to proccess the laptop drive from/on your HTPC computer. This would give you yet another dataset on how the app handles that drive(depending on your Shares and shared names an almost exact path situation) on you "problem" PC. EDIT:>I just looked at your logs and reread all your posts and saw that it's c: on your laptop. So It's unlikey that your sharing your root C: over your network(wouldn't be too wise, maybe even too dangerous to temporalily share for a few minutes just to run the tests.). Still you could likely safely go 1 step deeper into the path and make that a share, that you could then test on your HTPC <:EDIT I'm not sure if roxy99's app, explicitly was designed(However,from my experience with roxy99 I'd suspect this behavior was intentionl) for shared drives or UNC pathes but... I know shared drives worked on my network, at least using UNC style pathes. So it might be worth a shot to see what happens. Last edited by justme; 07-25-2007 at 11:07 AM. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
TV.Com to .My File Generator (MyScraper) | pdavis | SageTV Customizations | 18 | 02-07-2008 10:50 PM |
Automatic and integrated IMDB info for DVD on SageMC | mkanet | SageMC Custom Interface | 0 | 04-30-2007 12:02 PM |
TV.com import Info like IMDB Search | jbilich | SageTV Customizations | 1 | 08-10-2006 12:16 AM |
Automatic refresh? | afinemetsfan | SageTV Studio | 3 | 06-30-2006 09:29 PM |