Plugin: Sage Recording Extender (SRE)

[Slugger]

Originally Posted by kevine

I did not create an override. It did say "Teams TBA". Unsure about NFL Football because I erased it. You post above said it was fixed for the superbowl and does not mention an override is needed in that post. Therefore, I assumed that you were not just looking for teams for this but but for the word "superbowl".

I do stand corrected, I made this a separate post with a subject that would hopefully catch everyone's eye. It probably got bumped too far down for you to see it in time.

http://forums.sagetv.com/forums/showthread.php?t=30291

Originally Posted by kevine

I would think the log should contain only items that SRE took action on or did not take action on because it matched something in ESPN. In other words, If I did not schedule it to record, it should not log it. If there was not a match to my recording vs. what is in ESPN it should not log it. If that were the case my log would have 1 entry, had I put the override in.

Will probably move to this type of logging system in the next release.

Originally Posted by kevine

Isn't it also possible to know ahead of time if SRE found a matiching game with ESPN and will be monitored. "Yes, SRE found a game match for, NE Patriots vs NY Giants and will be monitored."

Thanks.

The plugin does this when you create overrides. If the override does not create a valid monitor it rejects it and tells you it's deleting the override. If you don't create an override then SRE doesn't go searching through every scheduled recording to see if there is a valid monitor. The main reason for this is to avoid traffic on ESPN's server unless absolutely necessary. It's been almost a year now and ESPN hasn't done anything to hinder this plugin (which would take little effort on their part), but if their traffic logs started filling up with hits from this plugin every 5 minutes (or hourly or whatever) then I'm sure they'd be more motivated to kill the plugin. Identifying the plugin hitting their web server is rather straightforward since JRE reports the browser as "Java 1.6" or something like that on each request. I highly doubt there are many others using Java programs to access ESPN content so one of my goals is to reduce the load on their servers to a minimum. The rule of thumb is, if the program has the proper title and the subtitle has the team names in it then SRE should be fine. If the subtitle doesn't describe who's playing then you need to create an override for the event. This link describes the required title/subtitle combinations for SRE to properly monitor an event.

[kevine]

In my overrides tab, every recording is listed. As a matter of fact, "Good Morning America" is being watched by my wife and in the Plug In Log tab, it says, "Good Morning America --> Too early to check status". Is this normal? Is it hitting the ESPN plugin when Good Morning America get close to its end?

[Slugger]

Originally Posted by kevine

In my overrides tab, every recording is listed. As a matter of fact, "Good Morning America" is being watched by my wife and in the Plug In Log tab, it says, "Good Morning America --> Too early to check status". Is this normal? Is it hitting the ESPN plugin when Good Morning America get close to its end?

No, since the program title doesn't match one that is to be monitored, it will never hit the ESPN servers. Once there is less than 30 minutes left in Good Morning America the plugin will say 'No monitor' instead of 'Too early to check status'. But, if I were to check the validity of programs to be monitored and notify users when the subtitle doesn't create a valid monitor then I'd be hitting the ESPN servers too often, plus there are other issues.

Let's forget the increased traffic at ESPN and say they don't/won't care. So I scan the upcoming recordings on every run. What happens if I notice a program has an invalid subtitle? Do I simply log this fact every 5 minutes and hope the user eventually reads the log file? Do I notify them with a JavaScript alert the next time they load the servlet? What if they never load the servlet before the recording? What if they've never even installed the servlet to begin with? I'm open to a more proactive approach to detecting and notifying users of programs that may need an override, but it's got to be something that they're guaranteed to see otherwise it doesn't justify the increased traffic on ESPN's servers.

My biggest concern is that there has to be a threshold at which point ESPN is going to notice a (bunch of separate instances of a) Java program constantly hitting their servers and decide they don't like it anymore. When that happens the plugin is broken. I don't mind fixing the parsers here and there, but if ESPN decides they don't want this program hitting their servers anymore then they could easily force a parser correction multiple times a week and when that happens I'm no longer able to maintain the plugin. To date, I've been fortunate enough not to have ESPN change the format of their pages even once (knock on wood). My goal is to keep it that way.

[kevine]

Originally Posted by Slugger

What happens if I notice a program has an invalid subtitle? Do I simply log this fact every 5 minutes and hope the user eventually reads the log file?

It should be ignored.

Originally Posted by Slugger

Do I notify them with a JavaScript alert the next time they load the servlet?

If I were writing this for myself and I don't know if this is even possible becasue I do not know how the plugin works.:
Daily:
1. SRE gets all of my recordings where the title matches 1 we are looking for (NFL Football, NHL Hockey, MLB, etc.) Whatever it looks for now to find a match.
2. SRE goes to ESPN and gets all the matching categories and the games, times etc. but for only the categories that matched in the above. For example, don't download NFL if I have no NFL games in my "to be recorded" list.
3. In the UI, display an icon. Again, don't know how this works but other plugins change the icons. For example, an icon box with SRE in it. If this is not possible, put it in the servlet. If someone wants that feature they will load the servlet and the webserver to match. In the servlet, simply put another column on the Override screen with "Yes" next to the ones it matched above.
4. When a game matches, SRE connects to ESPN as it does currently.

So SRE connects daily, when an override is chagned and when a game is in active monitoring of a game.

Originally Posted by Slugger

My biggest concern is that there has to be a threshold at which point ESPN is going to notice a (bunch of separate instances of a) Java program constantly hitting their servers and decide they don't like it anymore. When that happens the plugin is broken. I don't mind fixing the parsers here and there, but if ESPN decides they don't want this program hitting their servers anymore then they could easily force a parser correction multiple times a week and when that happens I'm no longer able to maintain the plugin. To date, I've been fortunate enough not to have ESPN change the format of their pages even once (knock on wood). My goal is to keep it that way.

I understand and concur. Who else uses this plugin besides us? Is there a way of creating different Java program names so they see different items connecting? (Daily, ative monitoring, etc.)

Can you point me to where this is at ESPN? If I can read up on it, I might be able to help suggest better. I am a programmer too.

[Slugger]

Originally Posted by kevine

Can you point me to where this is at ESPN? If I can read up on it, I might be able to help suggest better. I am a programmer too.

What info are you looking for? Basically I just download the scoreboard pages and parse the game data out of them.

For example, here is the NHL scoreboard I parse for NHL monitoring.

As to the usage, there are 400+ downloads of the plugin so I suspect there may be ~150 unique instances of the plugin floating around hitting the ESPN servers as necessary.

[kevine]

Originally Posted by Slugger

For example, here is the NHL scoreboard I parse for NHL monitoring.

OK. I see now.

OK, let's say I have an NHL game I want to record and it is scheduled for tomorrow. 2/9. SRE should get this page and see if it can find a matching game.

If it does mark it.


Also, Fox Sports website, MSNBC , etc. You can always go somethere else to get your data.

I can't see them changing it just to stop people from hitting it every 5 minutes. I know myself when I cannot be at a game, have hit refresh ever few seconds to make sure I knew what was going on. The only way they could stop you is changing it to a non-text based display. A Java control or something like that. But then users would need to download it. I don't see that happening.

[Slugger]

Originally Posted by kevine

OK. I see now.

OK, let's say I have an NHL game I want to record and it is scheduled for tomorrow. 2/9. SRE should get this page and see if it can find a matching game.

If it does mark it.


Also, Fox Sports website, MSNBC , etc. You can always go somethere else to get your data.

I can't see them changing it just to stop people from hitting it every 5 minutes. I know myself when I cannot be at a game, have hit refresh ever few seconds to make sure I knew what was going on. The only way they could stop you is changing it to a non-text based display. A Java control or something like that. But then users would need to download it. I don't see that happening.

Ok, marking a monitor as found and valid is straightforward, but that really isn't the issue. What happens if you have a game scheduled to record on Feb 9, but SRE isn't able to find data for the game? For example, the EPG data says "Teams TBA". This is really what needs to be flagged and the user notified about. Say we mark the Feb 9 game as having no data available today, but then tomorrow your EPG is updated to include the proper info. Now what? Do we kill the override if you created one and do the check again? Do we just ignore it since you've already created an override? If you didn't create an override do we keep checking ESPN and notifying the user? A no-match could be caused by ESPN (incorrect/missing data) so just checking once and marking something in the db probably isn't going to be enough and now we're back at hitting the ESPN site way more than is probably necessary. Admittedly, use of SRE does require a little manual intervention, but I think by doing it this way we get the best of both worlds - it tends to work most of the time with little or no intervention required and I minimize the traffic on ESPN's (or any source's) web servers.

[kevine]

I don't see where it should be on SRE or you to program it to let the user know of a Teams TBA problem. That is why you have an override. If I am interest in recording a game, I just want to know that SRE has found a match and is ready to start monitoring it. If it hasn't I create an override and hit a button to "check now" to make sure this game exists on ESPN and it is ready to start monitoring.

BTW, you have done a great job with this plugin. Like you said, it works most of the time. Better than what we had before (Nothing!).

[Slugger]

Originally Posted by kevine

I don't see where it should be on SRE or you to program it to let the user know of a Teams TBA problem. That is why you have an override. If I am interest in recording a game, I just want to know that SRE has found a match and is ready to start monitoring it. If it hasn't I create an override and hit a button to "check now" to make sure this game exists on ESPN and it is ready to start monitoring.

BTW, you have done a great job with this plugin. Like you said, it works most of the time. Better than what we had before (Nothing!).

I see where you're coming from now. Basically you want SRE to tell you if a program that is to be monitored can be monitored based on the available EPG data? If it can't, then you want SRE to highlight the program in red (or something) so that you know that an override will be required.

SRE already checks created overrides before accepting them to ensure that they are valid so the last part is already taken care of.

[Slugger]

SRE v1.5.3 is now available.

If you are attempting to monitor NCAA Men's basketball games from any of the post season tournaments (NCAAs, NIT, CBI) then you must upgrade to v1.5.3. ESPN changed the URLs used to access game information for the post season so all versions of SRE prior to 1.5.3 are broken for NCAA basketball monitoring of post season tournament games.

[Slugger]

v1.6.0 is now available.

Major highlight of this release is the introduction of the native Windows installer (XP and below) that will automatically install the SRE plugin and modify all config files to setup SRE. The installer will also setup and configure the SRE servlet for access within the web server plugin (optional).

v1.5.3 -> v1.6.0
================

* 1944467: Move location of static content to support proxy setups
* 1951282: Create Windows installer (for WinXP and below)
* 1957586: Move all logging to SRE database
* 1957588: Notify users of monitor status (via override table in servlet)

[Stuntman]

Had a game stop 4 minutes short today.. SRE didn't seem to catch the game as a NBA game and therefore didn't add any padding as I had it set to do... Is this perhaps something I can make adjustments for?

[Slugger]

Originally Posted by Stuntman

Had a game stop 4 minutes short today.. SRE didn't seem to catch the game as a NBA game and therefore didn't add any padding as I had it set to do... Is this perhaps something I can make adjustments for?

Which version of SRE are you running? Did you ensure that the EPG data listed the teams that were playing? I've been noticing that my EPG data (from Zap2It) has been listing many playoff games as:

Title: NBA Basketball
Episode: Playoffs: Teams TBA

In these cases, you will need to create an override to tell SRE which teams are playing in the game. Without an override, SRE cannot monitor games with such episode information. The latest version I uploaded last night (v1.6.0) would also point this out to you by highlighting the recording in red on the override tab of the servlet. Red means that SRE wants to monitor the recording, but can't and that you probably need to create an override.

[Stuntman]

Which version of SRE are you running?
I'm running the latest version..

I'm seeing the TBA too... I never had time to lookup the teams myself and didn't have access to SRE, so I went on the assumption that since SRE knew they were NBA games and that it knew I wanted them recorded.. that it would use my default padding on those games even if it couldn't determine what game it was. Would it be possible to make it work this way?

I've now padded my actual 'favorite' so that it should pad those recordings.

[Slugger]

Originally Posted by Stuntman

Which version of SRE are you running?
I'm running the latest version..

I'm seeing the TBA too... I never had time to lookup the teams myself and didn't have access to SRE, so I went on the assumption that since SRE knew they were NBA games and that it knew I wanted them recorded.. that it would use my default padding on those games even if it couldn't determine what game it was. Would it be possible to make it work this way?

I've now padded my actual 'favorite' so that it should pad those recordings.

Ah... I see what you're saying now. I will investigate this asap (but may not be for a day or two as work has got me tied up this week). Be warned that padding the favourite may (should) be overridden by SRE, if it's working as it's suppose to. Since it doesn't appear to be, then it might be that SRE is clearing the padding back to zero, which explains what you're seeing. If you can't setup the override for SRE then you may want to disable it so it doesn't (incorrectly) mess with default paddings until I can track down the problem.

[Slugger]

Originally Posted by Slugger

Ah... I see what you're saying now. I will investigate this asap (but may not be for a day or two as work has got me tied up this week). Be warned that padding the favourite may (should) be overridden by SRE, if it's working as it's suppose to. Since it doesn't appear to be, then it might be that SRE is clearing the padding back to zero, which explains what you're seeing. If you can't setup the override for SRE then you may want to disable it so it doesn't (incorrectly) mess with default paddings until I can track down the problem.

I found the cause of the bug, but will need a few hours to fix it. I am absolutely overloaded at work this week (they try to squeeze every last ounce out of you before vacation) so no promises, but I will try to post a v1.6.1 with this fix before I head out on vacation this weekend. If a fix isn't posted by Saturday morning then it won't be fixed until after Victoria Day (May 19). From what I'm seeing, a padding on the favourite should work fine as a temp work around in the case where the episode data is using 'Teams TBA' as the bug is that the default padding is being completely ignored in this particular case (as opposed to being reset or something like that).

[Slugger]

Originally Posted by Slugger

I found the cause of the bug, but will need a few hours to fix it. I am absolutely overloaded at work this week (they try to squeeze every last ounce out of you before vacation) so no promises, but I will try to post a v1.6.1 with this fix before I head out on vacation this weekend. If a fix isn't posted by Saturday morning then it won't be fixed until after Victoria Day (May 19). From what I'm seeing, a padding on the favourite should work fine as a temp work around in the case where the episode data is using 'Teams TBA' as the bug is that the default padding is being completely ignored in this particular case (as opposed to being reset or something like that).

Here is an updated jar file with the fix for bug 1958423. I won't have time to do a full 1.6.1 release until after I return from vacation, but to get this fix simply stop SageTV, replace the slugger_sage-recording-extender.jar (usually located in C:\Program Files\SageTV\SageTV\JARs\) file with this one, then restart SageTV. This jar file will upgrade your SRE to a temp version of 1.6.0a - the official 1.6.1 release, with this bug fix, will follow sometime after May 19.

NOTE: Only apply this jar file on top of an existing 1.6.0 installation. Using this jar file on top of anything prior to 1.6.0 will result in unknown/undefined/unsupported behaviour. Those wishing to wait for the full release to get this bug fix should follow the suggested work around mentioned above in this thread.

[Stuntman]

Originally Posted by Slugger

Here is an updated jar file

The link contains the uncompiled code.. not the jar file..

[Slugger]

Originally Posted by Stuntman

The link contains the uncompiled code.. not the jar file..

Try this link. Mind you, the link above also gave me the jar file, too. It's the second link above. The first links to the bug report at SourceForge and the second link links to the jar file.

[Slugger]

SRE v1.6.1 is now available.

v1.6.0 -> v1.6.1
================

* 1958423: Default padding not honoured for events with bad subtitles