Jetty 9 for SageTV 9

[stuckless]

Has anyone looked a creating a Jetty 9 plugin?

I know that we can't go to Java 8 in SageTV7 because of Jetty, but I haven't looked into why (I haven't had the time). At some point I will want to use Java 8 given that Java 7 is EOL.

I'm less concerned if things like sagex, bmt, mobile web, and neilm's web app, work on day 1 with Jetty 9, since I think they all need a refresh anyways.

I think we can pull in Jetty 9 as a new plugin, and give it a different configuration than the current (ie, port 8090 and use a different webapps configuration area, etc).

Personally, I'm looking at needing WebSocket support at some point, and I'll need Jetty 9 in order to get that.

Sagex Remote Apis would be easy to port to jetty 9 since it just registers a single servlet. BMT is all but abandoned... ie, I don't ever intend to start new work on it. I would consider collaborating on a Polymer based web ui that could probably end up replacing all 3 of the web UIs that we have today.

I might spend a few hours this weekend looking at Jetty 9 and see what's involved. I'm still actively working on the Android Placeshifter, but sometimes you just need a break (And change is sometimes as good as a break )

[Fuzzy]

Originally Posted by stuckless

Has anyone looked a creating a Jetty 9 plugin?

I know that we can't go to Java 8 in SageTV7 because of Jetty, but I haven't looked into why (I haven't had the time). At some point I will want to use Java 8 given that Java 7 is EOL.

I'm less concerned if things like sagex, bmt, mobile web, and neilm's web app, work on day 1 with Jetty 9, since I think they all need a refresh anyways.

I think we can pull in Jetty 9 as a new plugin, and give it a different configuration than the current (ie, port 8090 and use a different webapps configuration area, etc).

Personally, I'm looking at needing WebSocket support at some point, and I'll need Jetty 9 in order to get that.

Sagex Remote Apis would be easy to port to jetty 9 since it just registers a single servlet. BMT is all but abandoned... ie, I don't ever intend to start new work on it. I would consider collaborating on a Polymer based web ui that could probably end up replacing all 3 of the web UIs that we have today.

I might spend a few hours this weekend looking at Jetty 9 and see what's involved. I'm still actively working on the Android Placeshifter, but sometimes you just need a break (And change is sometimes as good as a break )

I remember reading that someone had try simply dropping it in, and it didn't work - but that's the extent of what I've seen. I really don't know what is involved in it.

I honestly wouldn't expect a newer jetty version to break the webapps - but you'd have to have the new Jetty working to start with.

[Slugger]

I briefly tried the jetty9 "drop it in and cross your fingers" experiment awhile back. From what I recall, the problem is jetty9 is servlet 3.0, but somewhere a servlet 2.5 implementation was still on the classpath, which caused problems. I think something other than the jetty plugin was providing a servlet 2.5 jar and I couldn't quickly track that down and then just never bothered to follow up.

The java 8 not working issue I believe comes down to the webui war calling deprecated apis that are officially removed in java 8 instead of just deprecated. Or the webui does not fully implement newer changes to interfaces updated in java 8. One or the other, can't remember, but I'm fairly certain all the java 8 issues are caused directly by the webui war only.

I thought I had a snapshot of a vm with all of my experiments on this subject, but I don't have a snapshot so all I have are my memories of what I tried and that was some time ago.

[stuckless]

I'm toying with the idea of running Jetty in a separate JVM. ie, the Jetty Plugin would have the ability to run in the current JVM (as it does now), or run in a separate JVM. Doing that would definately break the mobile and neilm apps. BMT and Sagex would still work, since sagex was primarily build to allow it to run remotely from a SageTV instance.

Running in a separate JVM would give the benefit of not having the web apps consume memory that sagetv could use, and/or, have the ability that if the web server is misbehaving, you can simply start/stop it from sagetv plugin interface.

[stuckless]

Originally Posted by Slugger

I briefly tried the jetty9 "drop it in and cross your fingers" experiment awhile back. From what I recall, the problem is jetty9 is servlet 3.0, but somewhere a servlet 2.5 implementation was still on the classpath, which caused problems. I think something other than the jetty plugin was providing a servlet 2.5 jar and I couldn't quickly track that down and then just never bothered to follow up.

The java 8 not working issue I believe comes down to the webui war calling deprecated apis that are officially removed in java 8 instead of just deprecated. Or the webui does not fully implement newer changes to interfaces updated in java 8. One or the other, can't remember, but I'm fairly certain all the java 8 issues are caused directly by the webui war only.

I thought I had a snapshot of a vm with all of my experiments on this subject, but I don't have a snapshot so all I have are my memories of what I tried and that was some time ago.

Thanks... I think if I do this, I'll make it co-exist with existing Jetty. I think the web apps need a complete refresh... so this way, jetty 9 can run along side of what is there, and if people want to move to Jetty 9, they can create a new app. ie, BMT is in dire need of a complete rewrite, and using polymer, if done right, it would actually run well on a desktop, and half decent on a phone/tablet.

[Slugger]

Hmm... not sure how well that's going to work. At the very least you're going to need to slap a great big, 40 point font warning that you won't be able to install both jettys together.

Jetty6 will include (somewhere, somehow) a servlet 2.5 implementation jar and jetty9 needs a servlet 3.0, but they both will collide (javax.http.servlet.** packages) if installed alongside each other.

Problems will really begin to surface when someone writes a new plugin and depends on jetty9 and a user installs that plugin, which installs jetty9, but then the same user finds another plugin that depends on jetty6 and installs that plugin. They end up with both jettys installed and the javax.http.servlet.** classpath collisions begin.

[Fuzzy]

This goes back to the need to 'adjust' some old plugins to add maxversions to allow forward movement without breaking old stuff. Get a wholly new version of the Jetty Plugin working, and then just see what webapps get broken by it, and either block them from using it (with maxversion tags on their jetty dependency) or fix them. I don't think having two Jetty plugins would ever be an ideal situation.

[stuckless]

Originally Posted by Slugger

Hmm... not sure how well that's going to work. At the very least you're going to need to slap a great big, 40 point font warning that you won't be able to install both jettys together.

Jetty6 will include (somewhere, somehow) a servlet 2.5 implementation jar and jetty9 needs a servlet 3.0, but they both will collide (javax.http.servlet.** packages) if installed alongside each other.

Problems will really begin to surface when someone writes a new plugin and depends on jetty9 and a user installs that plugin, which installs jetty9, but then the same user finds another plugin that depends on jetty6 and installs that plugin. They end up with both jettys installed and the javax.http.servlet.** classpath collisions begin.

Yeah, I haven't really looked at this hard enough to answer your questions... and likely you are right. There's always, the "What I'd like to do" vs "What I have to do"... They don't always align, so, if it doesn't work out, then that's ok as well... I'll go with what's in the best interest of the community.

The idea of running the jetty server in a separate VM would certainly mitigate all your concerns, but even if I allowed that... it wasn't meant to be the ONLY option, so I still have to deal with the possibility of having both.

And if turns out that we can't have both, we'll have to decide what to do.... at this point it's all theory.

[jreichen]

I did some work on this a few months ago and part of it is migrated. I replaced all the jars, updated all the code for the package and api changes, and updated all the xml files.

There are a couple unfinished items. One is JSP support. It's possible to get it working but I didn't get there. The /apps app and the Mobile Web Interface both use JSPs.

The other is that all the web plugins were required to deliver a context xml file that Jetty would read. All those files have Jetty 6 package and class name references in them. They were mainly used to set up security by app. I've never been crazy about doing it that way and would rather see centralized security.

I have a repo on GitHub that can be forked. I don't know if the latest has been pushed up but I could do that next week if anyone wants it.

As it stands right now, I don't see myself putting in a lot of dev time on SageTV. Maybe small contributions here and there, but not a major project. I am following the work going on, especially Android Placeshifter support, and hope to have a running Sage system again.

Jason

[GKusnick]

Originally Posted by Slugger

I'm fairly certain all the java 8 issues are caused directly by the webui war only.

For what it's worth, I've been running Jetty 6 with Java 8 without issues since Java 8 was released. But I don't use any of the standard Web UI plugins, just my own custom webapp.

[stuckless]

Originally Posted by GKusnick

For what it's worth, I've been running Jetty 6 with Java 8 without issues since Java 8 was released. But I don't use any of the standard Web UI plugins, just my own custom webapp.

Does your custom webapp use jsps?

Sounds like everyone is saying the same thing, really. ie, we don't have a Java 8 + Jetty problem, so much as we have a Java 8 + web ui issue.

When I looked at this, my thoughts were to treat the "jetty" part much like we do with comskip. ie, it has an installation area that we use/reference. For jetty, I thought it would be better to simply download the distrubution, as is, and reference that instead of putting all the jars in the JARs area. The Jetty Plugin part would use the Jetty Start that jetty provides to start/stop the server. We can pass the jars in the JARs area as part of the extended classpath to jetty, so that web-apps can reference the Sage apis. In this way, IF jetty were run stand-alone, it would only require simple change the arguments that you pass the Jetty Starter, since it already provides a mechanism to launch in its own jvm. Apps that use sagex would simply continue to work whether they were inside the sage jvm or running in a separate process. This is how I initially thought we could run Jetty6 and Jetty9 at the same time, since Jetty9 would be a completely different model, and not put any jars in the JARs folder.

For my needs, I'll eventually require WebSocket support. I don't need Jetty for this, but since Jetty 9 provides the WebSocket api, it would be easier.

@jreichen - if you have a repo for this, and you've already done some work, you might as well push it (maybe to a new branch) and I'll eventually take a look at it as well. FWIW, I never liked the Context xml files either,and I'd rather see a centralized security.

[GKusnick]

Originally Posted by stuckless

Does your custom webapp use jsps?

Yes.

[jreichen]

Here's the repo and branch with the Jetty 9 changes I was working on. The plugin should install and run in SageTV, but /apps doesn't work because JSP support isn't working and existing apps won't work because of their context xml files.

https://github.com/jreicheneker/Sage...er/tree/jetty9

Here are the Jetty 9 reference docs:

http://www.eclipse.org/jetty/documen...ent/index.html

Hope this helps.

[sdsean]

Btw I was just going to setup the old Jetty webserver plugin in my current SageTV 7 setup until I got things running with version 9, but the plugin seems to fail to download and the wiki's and whatnot are all gone. . .

Is there any way to get that back? at least the plugin and web server?

[stuckless]

Originally Posted by sdsean

Btw I was just going to setup the old Jetty webserver plugin in my current SageTV 7 setup until I got things running with version 9, but the plugin seems to fail to download and the wiki's and whatnot are all gone. . .

Is there any way to get that back? at least the plugin and web server?

The plugin, in the sagetv repository should be working. I installed this less than a week ago, and to my knowledge nothing has changed. Maybe just try it again.

[sdsean]

Originally Posted by stuckless

The plugin, in the sagetv repository should be working. I installed this less than a week ago, and to my knowledge nothing has changed. Maybe just try it again.

In the sagetv repository? Where exactly? I don't see anything in the GIT repro . . .

Although in fairness I'm talking about v7, not v9. . I meant that I"m trying to install the plugin from the UI in SageTV (v7), and it just utterly fails to download. . .

[stuckless]

Originally Posted by sdsean

In the sagetv repository? Where exactly? I don't see anything in the GIT repro . . .

Although in fairness I'm talking about v7, not v9. . I meant that I"m trying to install the plugin from the UI in SageTV (v7), and it just utterly fails to download. . .

I was meaning from the SageTV pluign UI as well. As I said, I installed it about a week ago.. but maybe something has changed since then.

[wayner]

I hope this doesn't break Slugger's plugins that use Jetty as well.

Is Jetty still the best way to implement a lightweight web server into SageTV?

[sdsean]

Hmmmm ok this stinks b/c it's for sure failing. . .

It tries to hit:

http://downloads.sourceforge.net/pro...ip?use_mirror=

And utterly fails. . .

So then I just opened up a browser. . .and sure enough it finds the download (at the above URL), and the project URL is here:

http://sourceforge.net/projects/sageplugins/

But then hitting :

http://sageplugins.sourceforge.net/

In that project page goes off to the assembla blog site that's busted.

So now the real question is. . . why is Sage failing to download the zip file when my browser does it just fine. . .

And. . .since I do now at least have the JAR files remind me how I can go install them manually?

[stuckless]

Originally Posted by sdsean

Hmmmm ok this stinks b/c it's for sure failing. . .

It tries to hit:

http://downloads.sourceforge.net/pro...ip?use_mirror=

And utterly fails. . .

So then I just opened up a browser. . .and sure enough it finds the download (at the above URL), and the project URL is here:

http://sourceforge.net/projects/sageplugins/

But then hitting :

http://sageplugins.sourceforge.net/

In that project page goes off to the assembla blog site that's busted.

So now the real question is. . . why is Sage failing to download the zip file when my browser does it just fine. . .

And. . .since I do now at least have the JAR files remind me how I can go install them manually?

While it is possible to manually install this, it's not trivial, since SageTV needs to "know" that it is installed. Typically you might do this by creating your plugins xml file and telling sagetv to use that. You can extract the files, but there are other things like plugin loader class,etc, that won't get updated if you just manually extract it.

This works in your browser because Sourceforge is redirecting your to landing page BEFORE autodownloading the file. No downloader is going to work well with that. Another reason to dislike sourceforge.

I think the maintainer should move the files to BinTray. The OpenSageTV organization has a sagetv-plugins area, and it's already being used by Phoenix and Gemstone.