Plugin: Sage Recording Extender 4 (SREv4)

[kchase56]

Looks like I'm only having an issue with MLB for some reason.
If I use http://localhost:8080/livepvrdata/events?date=20181125, I get the following results which seems to be correct for that date 11/25/2018 =>

COLLEGE BASKETBALL
==================
Florida State vs. Villanova Final
LSU vs. Oklahoma State Final
UAB vs. Canisius Final
Oral Roberts vs. James Madison Final/OT
UNCW vs. E Illinois Final
Duquesne vs. UMass Lowell Final
Cent Michigan vs. Siena Heights Final
Morgan State vs. Mt. St. Mary's Final
Utah vs. Northwestern Final
Fordham vs. Alabama A&M Final
Liberty vs. Savannah State Final
Illinois vs. Miss Valley St Final
Vermont vs. Bucknell Final
Quinnipiac vs. Maine Final
Columbia vs. St Joseph's BKN Final
Morehead State vs. Lipscomb Final
FAU vs. FGCU Final
SIU-E vs. Incarnate Word Final
Southern Miss vs. William Carey Final
Fairfield vs. Denver Final
Oakland vs. N Illinois Final
Bowling Green vs. Drexel Final
Fair. Dickinson vs. Lafayette Final
Wichita State vs. Rice Final
Gardner-Webb vs. Arkansas State Final
UNC Asheville vs. Furman Final
Cent Conn St vs. Pine Manor Final
Hawai'i vs. Fresno State Final
Long Beach St vs. Oregon State Final
Cleveland State vs. Purdue FW Final
SF Austin vs. St. Edward's Final
Seattle vs. Longwood Final
Richmond vs. Hampton Final
Brown vs. Bryant Final
Towson vs. Loyola MD Final
Charleston vs. Memphis Final
USC vs. CSU Bakersfield Final
Samford vs. SC State Final
Grand Canyon vs. La Salle Final
Seton Hall vs. Miami Final

COLLEGE FOOTBALL
================
No games for this date.

MLB BASEBALL
============
No games for this date.

FORMULA ONE RACING
==================
Abu Dhabi Grand Prix Race Complete

NASCAR RACING
=============
No games for this date.

NBA BASKETBALL
==============
Lakers vs. Magic Final
Pistons vs. Suns Final
Raptors vs. Heat Final
Grizzlies vs. Knicks Final
Hawks vs. Hornets Final
Nets vs. 76ers Final
Kings vs. Jazz Final
Trail Blazers vs. Clippers Final

NFL FOOTBALL
============
Bills vs. Jaguars Final
Bengals vs. Browns Final
Jets vs. Patriots Final
Eagles vs. Giants Final
Buccaneers vs. 49ers Final
Panthers vs. Seahawks Final
Ravens vs. Raiders Final
Chargers vs. Cardinals Final
Broncos vs. Steelers Final
Colts vs. Dolphins Final
Vikings vs. Packers Final

NHL HOCKEY
==========
Coyotes vs. Flames Final
Lightning vs. Devils Final
Predators vs. Ducks Final
Kings vs. Oilers Final

COLLEGE SOFTBALL
================
No games for this date.

COLLEGE BASEBALL
================
No games for this date.

WOMEN'S COLLEGE BASKETBALL
==========================
Hawai'i vs. Stanford Final
Fordham vs. Texas Final
Duke vs. Missouri Final
Miami vs. Temple Final
Richmond vs. Virginia Tech Final
Ball State vs. Quinnipiac Final
Bowling Green vs. Mercer Final
W Carolina vs. North Alabama Final
Rider vs. Boston College Final
Florida A&M vs. Omaha Final
Radford vs. Nebraska Final
Fair. Dickinson vs. Howard Final
Sacred Heart vs. Monmouth Final
Brown vs. Bucknell Final
Navy vs. NC A&T Final
Hofstra vs. Stetson Final
New Hampshire vs. Bryant Final
Cent Michigan vs. Chattanooga Final
IUPUI vs. VCU Final
Seton Hall vs. St. Francis BKN Final
UCF vs. Villanova Final
Saint Joseph's vs. Loyola MD Final
Norfolk State vs. Liberty Final
Tenn Tech vs. Old Dominion Final
FIU vs. Campbell Final
C. Carolina vs. Memphis Final
Appalachian St vs. Marshall Final
Butler vs. Illinois State Final
Ga Southern vs. Bethune-Cookman Final
St. Francis PA vs. Duquesne Final
UNC Greensboro vs. NC Pembroke Final
UNC Asheville vs. Wofford Final
Penn State vs. Stony Brook Final
James Madison vs. Winthrop Final
Binghamton vs. Army Final
Colgate vs. Columbia Final
Albany vs. Holy Cross Final
NJIT vs. SE Missouri St Final
Indiana vs. N Illinois Final
Colorado vs. Utah State Final
Colorado State vs. Cornell Final
Northwestern vs. UT Martin Final
Kansas vs. Iona Final
UT Rio Grande vs. St Thomas Hou Final
Auburn vs. Louisiana Final
LSU vs. Little Rock Final
Vanderbilt vs. Presbyterian Final
UMass vs. Fairfield Final
New Orleans vs. UL Monroe Final
Rice vs. McNeese Final
UTSA vs. Weber State Final
UCSB vs. Dartmouth Final
Maine vs. Penn Final
Wright State vs. Marist Final
Virginia vs. Saint Louis Final
Charleston So vs. Jacksonville Final
Lehigh vs. Charlotte Final
American vs. FGCU Final
Washington vs. Michigan Final
San Francisco vs. UT Arlington Final
Nevada vs. USC Final
UNLV vs. Middle Tenn Final
Pittsburgh vs. Tennessee State Final
Oakland vs. Canisius Final
Wisconsin vs. Arkansas Final

Any information on how to proceed debugging this issue would be greatly appreciated. I do have C++ programming experience, but haven't dealt with Java at all.

Thanks.

[kchase56]

The following line in EventMonitorMLB.groovy =>

static private final String FEED_URL = 'http://mlb.mlb.com/gdcross/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

needs to be changed to =>

static private final String FEED_URL = 'http://gd.mlb.com/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

Making this change locally and building a new livepvrdata.war file gives me correct results.

[emveepee]

Ah ok, that would be an ipv6 vs ipv4 issue.

Martin

[skiingwiz]

Originally Posted by kchase56

The following line in EventMonitorMLB.groovy =>

static private final String FEED_URL = 'http://mlb.mlb.com/gdcross/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

needs to be changed to =>

static private final String FEED_URL = 'http://gd.mlb.com/components/game/mlb/year_%s/month_%s/day_%s/master_scoreboard.json'

Making this change locally and building a new livepvrdata.war file gives me correct results.

When I go to the former, it redirects me to the latter, so there should be no difference.

Can you try going to http://mlb.mlb.com/gdcross/component...coreboard.json in your browser? It should redirect you to http://gd.mlb.com/components/game/ml...coreboard.json and return JSON data.

As with the others here, I'm not having any issues, so maybe it's something specific to your system. I even successfully recorded your Mets, since they were beating up on my Pirates.


Edit: Looking at the DNS, mlb.mlb.com routes through akamai whereas gd.mlb.com goes through cloudfront. Are you doing something to block akamai perhaps?

[kchase56]

Originally Posted by skiingwiz

When I go to the former, it redirects me to the latter, so there should be no difference.

Can you try going to http://mlb.mlb.com/gdcross/component...coreboard.json in your browser? It should redirect you to http://gd.mlb.com/components/game/ml...coreboard.json and return JSON data.

As with the others here, I'm not having any issues, so maybe it's something specific to your system. I even successfully recorded your Mets, since they were beating up on my Pirates.


Edit: Looking at the DNS, mlb.mlb.com routes through akamai whereas gd.mlb.com goes through cloudfront. Are you doing something to block akamai perhaps?

It does redirect me when using my browser.

I've reverted my changes locally and I've added some println statements and the following println =>

static final Event[] getFeed(Date date) throws IOException {
String dateStr = date.format('yyyy|MM|dd', FEED_TIME_ZONE)
String key = "mlbcom_$dateStr"
def events = AppRuntime.instance.statusCache.get(key)
if(events == null) {
def input = dateStr.split('\\|')
try {
def data = new URL(String.format(FEED_URL, input[0], input[1], input[2])).text
println "data = " + data

Produces the following in sagetv_0.txt =>

Wed 7/31 0:07:53.960 data = <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://gd.mlb.com/components/game/mlb/year_2019/month_07/day_31/master_scoreboard.json">here</a>.</p>
</body></html>



Does this help at all?

[skiingwiz]

Originally Posted by kchase56

It does redirect me when using my browser.

I've reverted my changes locally and I've added some println statements and the following println =>

static final Event[] getFeed(Date date) throws IOException {
String dateStr = date.format('yyyy|MM|dd', FEED_TIME_ZONE)
String key = "mlbcom_$dateStr"
def events = AppRuntime.instance.statusCache.get(key)
if(events == null) {
def input = dateStr.split('\\|')
try {
def data = new URL(String.format(FEED_URL, input[0], input[1], input[2])).text
println "data = " + data

Produces the following in sagetv_0.txt =>

Wed 7/31 0:07:53.960 data = <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://gd.mlb.com/components/game/mlb/year_2019/month_07/day_31/master_scoreboard.json">here</a>.</p>
</body></html>



Does this help at all?

That confuses me. According to the documentation, the Groovy URL object uses HttpURLConnection which uses the System Property http.maxRedirects which defaults to 20.

All of this matches up with what I see on my system. I can run just the URL line and get the correct answer, following the redirects.

Are you behind any kind of proxy, or something like that?

[kchase56]

Originally Posted by skiingwiz

That confuses me. According to the documentation, the Groovy URL object uses HttpURLConnection which uses the System Property http.maxRedirects which defaults to 20.

All of this matches up with what I see on my system. I can run just the URL line and get the correct answer, following the redirects.

Are you behind any kind of proxy, or something like that?

I am not behind any kind of proxy.

[kchase56]

If I run the following groovy script on the server machine, it works perfectly.
Could running the script through the Jetty server be causing this issue?

[skiingwiz]

While investigating this issue, I discovered that the gd.mlb.com data source is deprecated and they are directing everyone to move to a new statsapi.mlb.com with a different API and output format. So this issue really is moot.

I have the update ready to go, but I need some time to test it. I was thinking about not releasing it until after this MLB season is over. But if you'd like to test, you can download the war file here (in a zip). It seems to work fine locally, but I have done very little testing so far.

[macsupergeek]

Any chance the LLWS can be added? Looks like the scores are posted and updated via this website - https://www.littleleague.org/world-s.../world-series/

or maybe even a scrape of this one - looking for it to say final --
https://gc.com/tmt/summer-2019/littl...cb90002195a5da

I guess since its such a temporary and small event, maybe too much work?

[skiingwiz]

Originally Posted by macsupergeek

Any chance the LLWS can be added? Looks like the scores are posted and updated via this website - https://www.littleleague.org/world-s.../world-series/

or maybe even a scrape of this one - looking for it to say final --
https://gc.com/tmt/summer-2019/littl...cb90002195a5da

I guess since its such a temporary and small event, maybe too much work?

I won't have time to put this in place before the end of this year's tournament. I could get it in for next year, but I'll need a source that's better than just scraping a web site. Those are very likely to change and small changes can break the whole thing. All of the other monitored sports are hitting URLs that are returning some sort of structured data (usually JSON, but other formats would be viable, XML, YAML, etc.) If something like that exists for the LLWS, it would be very easy to add.

I'll add this to my list of ideas and see if I can find a data source. If anyone else knows of a source, speak up.

[cat6man]

this is my annual time to sing the praises of the best app EVER, SRE.

this year however, baseball's opening day has been postponed, but i'll note that the one pre-season red sox game i recorded was, in fact, picked up and tracked by SRE.

stay safe and healthy everyone, and remember to help your neighbors whenever you can.

[dlj9999]

Hello,

The WNBA starts today(July 25) but i dont see it listed on livepvrdata.
Could this be added to the list?

Thank you.

[skiingwiz]

Originally Posted by dlj9999

Hello,

The WNBA starts today(July 25) but i dont see it listed on livepvrdata.
Could this be added to the list?

I can take a look at adding this. I'll have to dust off the development environment.

[wayner]

FYI, NBA and MLB games tonight are being monitored, NHL game(s) in the next few days show as being monitored. Great to see that this is still working!

[skiingwiz]

Version 1.8.0 adds support for WNBA monitoring. It'll be available as soon as the pull request here is merged into the SageTV Plugins repo. (Which has to be done by someone other than me.)

I had forgotten how simple it was to add a new monitor to this code as long as the data is there to support it.

[dlj9999]

Originally Posted by skiingwiz

Version 1.8.0 adds support for WNBA monitoring. It'll be available as soon as the pull request here is merged into the SageTV Plugins repo. (Which has to be done by someone other than me.)

I had forgotten how simple it was to add a new monitor to this code as long as the data is there to support it.

Thank you so much!!

[wayner]

A busy day for SRE!

Today is the first day EVER that the NFL, MLB, NBA, WNBA, MLS and NHL will all play on the same day Exploding head.

Here is my page from LivePVR data:

COLLEGE FOOTBALL
================
Miami vs. UAB Sep 10 20:00 ET

MLB BASEBALL
============
Detroit vs. St. Louis In Progress
Detroit vs. St. Louis Preview
Houston vs. Oakland In Progress
LA Angels vs. Texas Warmup
Atlanta vs. Washington Pre-Game
Kansas City vs. Cleveland Pre-Game
Boston vs. Tampa Bay Pre-Game
Philadelphia vs. Miami Pre-Game
Baltimore vs. NY Yankees Pre-Game
Cincinnati vs. Chi Cubs Pre-Game
San Francisco vs. San Diego Pre-Game
LA Dodgers vs. Arizona Preview

NBA BASKETBALL
==============
Rockets vs. Lakers Sep 10 19:00 ET

NFL FOOTBALL
============
Chiefs vs. Texans Sep 10 20:20 ET

NHL HOCKEY
==========
Stars vs. Golden Knights Sep 10 20:00 ET

[wayner]

I am restoring my old SageTV server that has been dead for six months due to a bad hard drive. This will be used as a backup.

I reinstalled Win10 32 bit, and 32 bit SageTV from scratch and installed the Jetty web server, the web UI, SRE and Liveprrdata. The Web UI works properly, but when I go to the livepvrdata URL I just see this:

Directory: /livepvrdata/
META-INF/ 	0 bytes 	19-Dec-2020 4:22:33 PM
WEB-INF/ 	0 bytes 	19-Dec-2020 4:22:33 PM

edit - One thing that looks weird compared to my other servers is that in the C:\Program Files\SageTV\SageTV\jetty\webapps folder there is a livepvrdata folder and a livepvrdata_1 folder. My other servers don't have that.

[wayner]

Maybe this is just a temporary issue but I am not showing any games for today. There should at least be NHL and NBA games, and probably college Basketball.