Plugin: Sage Recording Extender 4 (SREv4)

[pvr599]

Worked absolutely PERFECT tonight. Thank you SO much Slugger. You rock!!

[Slugger]

So, as promised, I've been working away at a complete OSS port of the livepvrdata.com web service and I've got something up and running. It isn't complete, but SRE users can start doing some testing for me to make sure I haven't broken anything.

All I have is a test driver that contains the same client library that SRE is using, so in theory if my driver works then so should SRE, but theory is, well, just that.

Now some background on what I'm doing/planning to deliver...

First, my OSS port is written in Java + Groovy. The existing service was Java, my monitors are Groovy so this was the path of least resistance for me. The final product will be delivered as a docker container that can then be deployed on any docker host in the cloud. I'm currently using IBM's Bluemix cloud platform, which is what I will deliver and leave running until my trial runs out, at which point someone here will have to decide what to do next. The entire code base is now at github. (Well, it will all be there eventually -- as I get the ported code to compile, I will push it up to github.)

The community here is free to do what you want with the code/container/etc. Maybe you decide to port it to another language like php to then host on a cheaper hosting platform, etc. Up to you guys. You'll notice the github repo is attached to an org called 'livepvrdata-oss'. Obviously, I'm the only member right now but my expectation is that those interested in keeping this service/plugin alive will join the org and take it over. Anyone wanting to join now can, just pm me your github id.

The port is the bare minimum required to allow the existing SRE plugin (plus a couple other clients outside of Sage) to continue to monitor events. I have not (and will not) port any of the online admin functions. The vendor lock is too deep along with the fact that I'm not all that enthusiastic about doing this port as I don't use any of this anymore. Unfortunately, I can't just dump any of the source to gitlab for others to pick up as it's not mine to open source. I will provide a readme or something in the repo outlining what I didn't port and how one might go about bringing such features back to life, if so desired.

As mentioned, this app has been running on Google's appengine and the existing code base is heavily vendor locked to those apis. The port removes all vendor lock and delivers a docker container using standard(ish) tools and frameworks. Given I'm only porting the bare minimum, it's actually quite simple:

* Java 7 (might move to Java 8 before I'm done)
* Groovy 2.3.x (might move to 2.4.x before I'm done)
* Jetty 9.x app container (replaces appengine runtime)
* Apache Derby [embedded] (replaces appengine datastore)
* Ehcache (repalces appengine memcache)

The docker container runs jetty with a war of the app inside of it. I chose to include an embedded SQL database in case the community decides to reimplement things like the web based admin functions, etc. In that case, a proper database will make life easier.

The current version of the port is running in the cloud as we speak:

http://134.168.0.29

In theory, SRE users could update their plugin settings and point it to that url and if I'm on the right track, all should work fine. I will only be porting the big 5 monitors (MLB, NFL, NBA, NHL, NCAAB/NCAAF). I will dump all the monitors I've written into github, but they may or may not need some attention.

Finally, the google appengine app will be deleted at any point after May 31, 2016. I will mark it for deletion on May 31 and then google will say they will remove it at any time after that, without warning. If I were to tweak or fix anything, it'd be with the full OSS code base. In reality, after I finish this initial port, I don't plan on having much to do with any of this code. After May 31, I'm just toasting the old code repo and walking away from the appengine version of this service. The livepvrdata.com domain name will expire on May 10, but that shouldn't affect SRE users that have upgraded to the latest version.

By June 1, the community will need to decide where they want to host the web service, if you choose to keep this plugin alive. The Bluemix cloud is pretty decent as it just lets you run your docker containers. They have a free tier that seems to be capable of hosting this service, but after the trial period, you have to put a credit card on file that they can charge in case you exceed the free tier. I'm not willing to do that, especially for a service that I do not use and do not closely monitor. Someone else here may be ok with that or perhaps other ideas. Either way, something will need to happen by June 1 to keep the current version of the SRE plugin operational.

[skiingwiz]

Very cool. Thanks for the time you've put into this. I haven't done much yet buy clone the repo, but I wanted to make sure to say thanks.

[jdezur]

I too am very grateful for all that you've done Slugger. I've been using Sage since v5, and SRE since v2. Every time I've tried to change PVR software, it just seems so clunky or missing something compared to the features that sage has. I hope to run it as long as it works, and when things stop working, I'll probably cut the cord to my hdhomeruns. We never watch live TV any more, even sports. SRE has been wonderful and I hope someone picks it up and keeps it going. I'd love to do it since it's of so much value to me, but I don't know squat about java or programming. I'm pretty useless outside of being a diehard user.

It doesn't sound like there's a huge crowd still using it, but I'm glad you've continued to support it. So many devs just code & run, leaving their project orphaned and broken. You, sir, are a true gentleman in my book. Thank you for your years of support.

[jbuszkie]

I am also very grateful for your continued support! I am a very infrequent user of SRE (Football season and if my baseball teams get into playoffs) but I applaud SLUGGER every time I go use it! Thanks, bud!

jim

[Slugger]

So my port of the web service is complete. As I said, this is the bare bones minimum port that will allow the SRE plugin to keep monitoring events.

The source code is here.

People are going to need to either join this github org and take over this existing repo or someone will have to fork it and run with it.

I've started documenting what I did and didn't port, some issues I've run into deploying it on the IBM cloud (and how someone may go about fixing those issues if you decided to keep running it on the IBM cloud), etc., etc. The docs aren't complete, but I'll attempt to finish them over the next week or two -- nudge me if you aren't noticing any updates to the wiki. Those docs are being written here.

I have been testing against the test driver I had lying around, which was compiled against a client lib that should be equivalent to what SRE is running. This means that SRE should be able to monitor against the deployed version of the app (details on its location are above). However, that deployed docker container will only remain up for another 19 days then my trial period ends on the bluemix cloud and I will not be keeping it running.

Anyone who decides to take this over should attempt to build and run the app and ask any questions about how to do those things, if necessary. If you ask questions, I'll try to document answers in the wiki.

That's about it for me on this project... I'll finish up the docs, but the code is done as far as I'm concerned. I doubt the code is perfectly ported (i.e. bug free), but it's certainly a starting point for the community to take over. I didn't port any of the unit or functional tests as I got sick of looking at google apis. Instead I created a test driver that basically just lets you run the app locally in an embedded jetty server. You can then debug within your IDE as necessary. Not ideal, as it basically relies on manual testing, but, honestly, this bare minimum port is so small that it's probably good enough. With that said, I did leave references to some test tools in the build file that someone may want to leverage if you wanted to add some unit tests or some automated functional tests.

Enjoy...

[Slugger]

My OSS port notes are complete. My work is done. If there's something someone needs to know then PM me otherwise, I'm done with this project.

I'll post some final warnings when the domain expires and when I mark the appengine version of the app for deletion (May 2016).

Also, the current deployment of the OSS version I have running on the IBM cloud will die in a couple of weeks when my trial period expires.

[cat6man]

hi slugger

thanks for your great work on the port.
i got my linux v9 server up and working this weekend for the first time and will test out your ported stuff in the next week.

also, since it is opening day, it is time for my annual shout out to slugger for my all time favorite app (SRE)...........absolutely brilliant concept and execution!



happy opening day everyone!

[Slugger]

Times are good up north when the Blue Jays are featured on ESPN's opening day coverage! Feel like they're either going to have to get out of this world pitching from the kids in the rotation or pull a rabbit out of the hat like Double A did at last year's deadline to be in playoff contention again, but definitely excited for another season to start!

[skiingwiz]

Originally Posted by Slugger

The port is the bare minimum required to allow the existing SRE plugin (plus a couple other clients outside of Sage) to continue to monitor events.

What are the other clients? I noticed that the client repo has a .Net client as well.

[Slugger]

I think NextPVR was the one using the .net client for awhile -- might still be using it. There were also some individuals using both clients for research/PoC stuff at various times. I think someone using MediaPortal was also looking at the .net client once upon a time but didn't ever do anything with it.

[Slugger]

Might also be worth noting that Schedules Direct offers a similar service these days, too. If you have an account with them, you can poll their service to get similar type responses. The apis are different though, but it's documented somewhere in their api docs. Can't really write a Sage plugin based on it though as it would only be useful for those with Schedules Direct creds.

[skiingwiz]

We, as a community, need to decide what to do with what Slugger has given us. If anyone knows of free hosting for java apps, I'm willing to do the work to get this running on that hosting.

If not, there are a couple of possible options. I have the livepvrdata server running in Jetty inside of sage. That's not an ideal way to do it because everyone is running their own servers then. That negates what I presume are the reasons Slugger created the web service first place. It would at least keep SRE running. I might consider doing this as a stopgap if we get near the deadline and don't have an alternative.

Another possible option is a few volunteers to run servers. I could update the client to pull down a list of known servers and round-robin among them.

A third option is to move away from the web service entirely. One of the big advantages of the web service is the ability to update monitors without having to push an update of the plugin. There are other ways to do that. Hand-waving some of the details for now, scripts for parsing the responses could be pulled down from a server and run locally. That gets us to only needing to host static files. (There would be code-signing involved and other security concerns, but as I said, hand-waving for now.)

I'm open to any and all suggestions. It would be nice to find a way to host this service mostly as-is, but if not I'm determined to get something done.

[Slugger]

Reminder: The livepvrdata.com domain has now expired. The web service is now only available at livepvrdata-hrd.appspot.com. A further reminder that this version of the web service is no longer supported and will be marked for deletion at Google Appengine within the next month. I will post another message when I've marked the service for deletion.

[jbuszkie]

Originally Posted by skiingwiz

Another possible option is a few volunteers to run servers. I could update the client to pull down a list of known servers and round-robin among them.


I'm open to any and all suggestions. It would be nice to find a way to host this service mostly as-is, but if not I'm determined to get something done.

I would be willing to host a server. I have a machine that's on 24/7. I'm not sure what I would have to do.. someone would have to walk me through it. but as long as I don't get a s-load of traffic, I'd be willing. There can't be too many of us that use this. I only use it for NFL and the occasional baseball or hometeam playoff game. So my use is sporadic.

Jim

[skiingwiz]

I just released a version of the Live PVR Data Server that runs insides the Sage Jetty instance. To try it out, install the Live PVR Data Server plugin within the SageTV Plugin manager and then configure SRE to use the server URL "http://localhost:8080/livepvrdata".

I've been using this for the last few days and successfully monitored several MLB games.

You can see the current days events by going to http://localhost:8080/livepvrdata/events

Note that if you run Jetty on a non-default port, you may need to adjust the URLs above.

Please try it out and report any problems in this thread. This may not be the final solution, but it will allow SRE to work beyond the end of Slugger's server.

[KarylFStein]

I installed and have a baseball game recording tonight. It says it's being monitored, so we'll see.

I'm also willing to host a server if this moves to a "crowd" approach.

Note that if you go to http://localhost:8080/apps, there is a new (duplicate) entry called "SageTV Web Interface" which is actually a link to the livepvrdata service.

EDIT: I may have missed the instructions on how to change the URL, so a refresher for others who can't find it: it's in the Sage.properties file, (shut down the SageTV service before editing).

sre4/clientUrl=http\://localhost\:8080/livepvrdata

[skiingwiz]

Originally Posted by KarylFStein

Note that if you go to http://localhost:8080/apps, there is a new (duplicate) entry called "SageTV Web Interface" which is actually a link to the livepvrdata service.

Thanks for noticing that. That will be fix for the next update.

Originally Posted by KarylFStein

EDIT: I may have missed the instructions on how to change the URL, so a refresher for others who can't find it: it's in the Sage.properties file, (shut down the SageTV service before editing).

sre4/clientUrl=http\://localhost\:8080/livepvrdata

You can also configure this by going to the plugin manager within the SageTV interface, finding Sage Recording Extender, and selecting it. When you do that, you should see an option to configure the plugin. The very last setting is the Server URL. (Though editing the properties file is probably easier, as long as Sage is shut down when you do it.)

[KarylFStein]

Originally Posted by skiingwiz

Thanks for noticing that. That will be fix for the next update.



You can also configure this by going to the plugin manager within the SageTV interface, finding Sage Recording Extender, and selecting it. When you do that, you should see an option to configure the plugin. The very last setting is the Server URL. (Though editing the properties file is probably easier, as long as Sage is shut down when you do it.)

That's the first thing I tried. But I must have been looking at the wrong thing as now I see the configuration option.

It doesn't seem like things worked. The recording was 3hrs exact and the logs had several messages like this, (two minutes apart from start to end):

2016-05-16 22:00:00,000 INFO : 24443637/MLB Baseball: Fetching status with data: [MLB Baseball, Minnesota Twins at Detroit Tigers, 1463439600000]
2016-05-16 22:00:00,108 WARN : 24443637/MLB Baseball: Error from web service; safety measures already applied.

EDIT: I see connections from the Jetty logs:

127.0.0.1 -  -  [17/May/2016:02:00:00 +0000] "GET /livepvrdata/query?q=%7B%22type%22%3A%22MLB+Baseball%22%2C%22details%22%3A%22Minnesota+Twins+at+Detroit+Tigers%22%2C%22start%22%3A1463439600%7D HTTP/1.1" 200 4043 "-" "SageTV_SRE/4.0.1.20160216213041"

Manually trying the URL in a browser returns this as the first line:

{"message":"Unexpected error","stackTrace":"java.io.IOException: Unable to find event data! [Minnesota Twins at Detroit Tigers]

[wayner]

Thanks, it is working here - even with a manual override to correct team names that aren't in the EPG yet:

	NBA BASKETBALL
	==============
	Cavaliers vs. Raptors                             	May 17 20:30 ET