Remote SageClient Possible?

[tmiranda]

One of the issues I am having with my Echo skill is the latency involved with using sagex to fetch information from the Sage server. One way to solve this would be to have a synced copy of the wiz.bin on the same web server as the Echo skill backend.

I know the SageClient does this already but it only works on a local network. Placeshifter works over a network but doesn't keep a local copy of the wiz.bin. Would it be possible to combine the two features?

No UI would be needed.

I can see this functionality being useful for other applications. If anybody decided to integrate Sage with Google Home for example.

Thoughts?

[stuckless]

Originally Posted by tmiranda

One of the issues I am having with my Echo skill is the latency involved with using sagex to fetch information from the Sage server. One way to solve this would be to have a synced copy of the wiz.bin on the same web server as the Echo skill backend.

I know the SageClient does this already but it only works on a local network. Placeshifter works over a network but doesn't keep a local copy of the wiz.bin. Would it be possible to combine the two features?

No UI would be needed.

I can see this functionality being useful for other applications. If anybody decided to integrate Sage with Google Home for example.

Thoughts?

Have you verified that you are dealing with a latency between the Echo Server and the SageTV Server, not a latency between the Echo client and the Echo Server?

I have an Echo Dot, and when I walk into a room, and say, "Alexa, turn on the light", there's about 1-2 second delay before the light comes on, and these things are on the same home network.

I'm not familiar with the architecture of how you've integrated Echo and SageTV, but, in general, on the same network, sagex REST apis are extremely fast... in many cases less than 50ms-100ms. If the data coming back is 1000s of movies, then you'll get some delay there, because the size of the message is so large. But for messages in the 1-5k range, those should be almost instant.

Do you have some examples of APIs that are taking a long time?

It takes a fair bit of work to create a data sync and keep things in sync, and do it efficiently. I'd hate to see you spend 2-3 weeks doing it, only to discover there is only a marginal performance increase, with huge code complexity.

[tmiranda]

Originally Posted by stuckless

Have you verified that you are dealing with a latency between the Echo Server and the SageTV Server, not a latency between the Echo client and the Echo Server?

I have an Echo Dot, and when I walk into a room, and say, "Alexa, turn on the light", there's about 1-2 second delay before the light comes on, and these things are on the same home network.

I'm not familiar with the architecture of how you've integrated Echo and SageTV, but, in general, on the same network, sagex REST apis are extremely fast... in many cases less than 50ms-100ms. If the data coming back is 1000s of movies, then you'll get some delay there, because the size of the message is so large. But for messages in the 1-5k range, those should be almost instant.

Do you have some examples of APIs that are taking a long time?

It takes a fair bit of work to create a data sync and keep things in sync, and do it efficiently. I'd hate to see you spend 2-3 weeks doing it, only to discover there is only a marginal performance increase, with huge code complexity.

I'm sure it will help because I simplified my example. There are actually several areas where it will help:

- To implement some functionality I need to make many (possibly hundreds, or thousands - it depends on the number of Airings in the DB) of sagex calls. This slows things down a lot.

- I'm using Amazon's Lambda to process requests and it's totally stateless. The only way to pass anything but the simplest pieces of info between user utterances is to save it to the DynamoDB, which is slow. If I had the DB locally I could quickly rebuild the data I need.

- Passing large numbers of complex objects (like Airings) is slow. (I tried only passing the fields I need instead of the whole Airings but this is only marginally better since I need many of the fields.)

What I am doing now works but has limitations. I currently save lists of AiringIDs to a DynamoDB and then when I need to get detailed information I get the Airing Object from the server.

[Narflex]

And you can run SageTVClient over the Internet...you just need to have the right ports open to do it (the main downside being there is no security preventing somebody else from connecting; so it'd probably be a better idea to do something like port forwarding over SSH). You'll need port 42024...and if you want to play stuff back, you'll also need 7818 for streaming. I *think* that should do it. I know I haven't tried that in a long time. Performance likely may not be that great since there'll be a decent amount of latency in server RPCs and we never designed SageTVClient to perform well in that kind of scenario (but with the V9 changes, this client won't slow down what's happening on the server which would have occurred in V7).

[stuckless]

Originally Posted by tmiranda

I'm sure it will help because I simplified my example. There are actually several areas where it will help:

- To implement some functionality I need to make many (possibly hundreds, or thousands - it depends on the number of Airings in the DB) of sagex calls. This slows things down a lot.

- I'm using Amazon's Lambda to process requests and it's totally stateless. The only way to pass anything but the simplest pieces of info between user utterances is to save it to the DynamoDB, which is slow. If I had the DB locally I could quickly rebuild the data I need.

- Passing large numbers of complex objects (like Airings) is slow. (I tried only passing the fields I need instead of the whole Airings but this is only marginally better since I need many of the fields.)

What I am doing now works but has limitations. I currently save lists of AiringIDs to a DynamoDB and then when I need to get detailed information I get the Airing Object from the server.

I guess I'm trying to envision "why" the echo would need to know about 1000s of airings, instead of just a few small details. Unless you are saying, "Alexa, tell me the next 1000 recordings", why would it need to know so much? I guess what I'm saying/asking is why isn't the server doing more work there, where it has all the data, and only sending back a small response? When I was doing Phoenix Android client, there were many times that I just registered a sagex calls on there server and hard it process the complex request request over 1000s of objects instead of getiting 1000s of objects and then making one or more calls for each to do an operation.

[tmiranda]

Originally Posted by stuckless

I guess I'm trying to envision "why" the echo would need to know about 1000s of airings, instead of just a few small details. Unless you are saying, "Alexa, tell me the next 1000 recordings", why would it need to know so much? I guess what I'm saying/asking is why isn't the server doing more work there, where it has all the data, and only sending back a small response? When I was doing Phoenix Android client, there were many times that I just registered a sagex calls on there server and hard it process the complex request request over 1000s of objects instead of getiting 1000s of objects and then making one or more calls for each to do an operation.

I allow the user to say a series of utterances like:
- "Alexa, open SageTV"
- "Get all recordings"
- "Filter out the watched recordings"
- "Group them by title"
- "Sort them the episodes by date recorded"

This is just an example of how a user could build a "virtual view". The point is there are lots of cases where I am working with many Airings.

[tmiranda]

Originally Posted by Narflex

And you can run SageTVClient over the Internet...you just need to have the right ports open to do it (the main downside being there is no security preventing somebody else from connecting; so it'd probably be a better idea to do something like port forwarding over SSH). You'll need port 42024...and if you want to play stuff back, you'll also need 7818 for streaming. I *think* that should do it. I know I haven't tried that in a long time. Performance likely may not be that great since there'll be a decent amount of latency in server RPCs and we never designed SageTVClient to perform well in that kind of scenario (but with the V9 changes, this client won't slow down what's happening on the server which would have occurred in V7).

Cool. I'm not interested in actually playing anything, I just want access to a local wiz.bin that synced to the server.

[stuckless]

Originally Posted by tmiranda

I'm sure it will help because I simplified my example. There are actually several areas where it will help:

- To implement some functionality I need to make many (possibly hundreds, or thousands - it depends on the number of Airings in the DB) of sagex calls. This slows things down a lot.

- I'm using Amazon's Lambda to process requests and it's totally stateless. The only way to pass anything but the simplest pieces of info between user utterances is to save it to the DynamoDB, which is slow. If I had the DB locally I could quickly rebuild the data I need.

- Passing large numbers of complex objects (like Airings) is slow. (I tried only passing the fields I need instead of the whole Airings but this is only marginally better since I need many of the fields.)

What I am doing now works but has limitations. I currently save lists of AiringIDs to a DynamoDB and then when I need to get detailed information I get the Airing Object from the server.

Originally Posted by tmiranda

I allow the user to say a series of utterances like:
- "Alexa, open SageTV"
- "Get all recordings"
- "Filter out the watched recordings"
- "Group them by title"
- "Sort them the episodes by date recorded"

This is just an example of how a user could build a "virtual view". The point is there are lots of cases where I am working with many Airings.

I can see that, but I guess even in those cases, I would have still opted to do all that work on the server and just let alexa store a token (session) and then keep passing that token in each command. At the end of the day, unless I'm totally missing what alexa is doing, those recordings are still on the sagetv server, and alexa isn't reading back 1000s of recordings at the end of each command.

[tmiranda]

Originally Posted by stuckless

I can see that, but I guess even in those cases, I would have still opted to do all that work on the server and just let alexa store a token (session) and then keep passing that token in each command. At the end of the day, unless I'm totally missing what alexa is doing, those recordings are still on the sagetv server, and alexa isn't reading back 1000s of recordings at the end of each command.

There is a Session Object that gets passed to the skill, but it has a size limit and I exceed limit when processing a large number of recordings.

[stuckless]

Originally Posted by tmiranda

There is a Session Object that gets passed to the skill, but it has a size limit and I exceed limit when processing a large number of recordings.

I think you misunderstand. I'm not suggesting that you store the recordings in the Alexa session, but that you use the Alexa session a unique token on the server to hold onto information.

In my understanding, Alexa, is just remote controlling SageTV.

So in the case of what you described, no large amounts data need to be transferred to alexa...

You - "Alexa, Open SageTV"
Server - gets "session id" (not sure what else "open sagetv does")
Alexa - "ok"

You - "Get All Recordings"
Server - gets all recording and stores them under the session id on the server. returns "ok"
Alexa - "ok"

You - "Filter out watched recordings"
Server - uses the session id to find the list of recordings, then filters by watched, stores result in the server for the session id. Returns "ok"
Alexa - "ok"

You - "Group them by title"
Server - uses the session id to find the list of filtered recordings, groups them, puts then in the session on the server. Returns "ok"
Alexa - "ok"

You - "Sort them the episodes by date recorded"
Server - uses the session id to find the list of grouped recordings, groups them, puts then in the session on the server. Returns "ok"
Alexa - "ok"

You - "What is next recording?"
Server - uses the session id to find the list of recordings, gets the first recording, sends it back. Returns JSON airing details for that ONE recording.
Alexa - "The next recording is Colony, season 2 episode 2. In this episode ...."

You - "Play the next recording?"
Server - uses the session id to find the list of recordings, gets the first recording, tells sagetv to play it. return "ok"
Alexa - "ok"

So, in this dialog... until alexa needs to actually speak something about the data, nothing is really returned back, except that it worked. All the data is held on the server, and in each case, Alexa is just sending it's session id, so that you can find the data associated with the session, but nothing more than a few bytes is ever sent, either way.

[tmiranda]

Originally Posted by stuckless

I think you misunderstand. I'm not suggesting that you store the recordings in the Alexa session, but that you use the Alexa session a unique token on the server to hold onto information.

...

So, in this dialog... until alexa needs to actually speak something about the data, nothing is really returned back, except that it worked. All the data is held on the server, and in each case, Alexa is just sending it's session id, so that you can find the data associated with the session, but nothing more than a few bytes is ever sent, either way.

OK, now I see what you are talking about. I never considered storing the info in the Sage server, I didn't think that was possible using sagex and JavaScript. I assumed, maybe incorrectly, that the JavaScript that sagex runs is also stateless.

How would I store the information using JavaScript? Can I declare static variables that are preserved for as long as the Sage server is running?

[stuckless]

Originally Posted by tmiranda

OK, now I see what you are talking about. I never considered storing the info in the Sage server, I didn't think that was possible using sagex and JavaScript. I assumed, maybe incorrectly, that the JavaScript that sagex runs is also stateless.

How would I store the information using JavaScript? Can I declare static variables that are preserved for as long as the Sage server is running?

This would require that your Alexa server component was a Java plugin. You could still use javascript for some parts of it, and have it call into the plugin, or just ditch javascript totally (I'd do the latter ).

You can still do it all in JS, but it would likely require that you at least have 1 small java api that allows you to statically set a name value pair in static memory, because you are right, the JavaScript calls themselves are, for the most part, stateless.