Feature Request: Accessing non-postal code based lineups

[pendragonsound]

I've been working with SD to help setup meaningful FTA lineups. But if I'm going to use them in Sage, it appears the existing Sage SD UI may not work because the "ZZZ" lineups cannot be accessed through a postal code filter. I would need to manually enter the names. Is there a way to get around this, like massaging a properties file, or will this require a code change? If the latter, would this be feasible? Thanks!

[EnterNoEscape]

It looks like something unexpected it being returned by ZZZ. It's purely accidental that it doesn't skip the postal code screen, but it shouldn't prevent things from working anyway. I'll look into it.

[rkulagow]

In some areas of the world it's easier to just put in the lineupID if it's known. For example, because of how postal code coverage works in places like the UK, your "best" transmitter (as known by "you") may not be in your postal code for whatever reason. That's why we publish

https://json.schedulesdirect.org/20141201/available

Which you can see has a dvb-t REST endpoint available (currently only GBR, but we had AUS and NZL back when we were still experimenting with them)

https://json.schedulesdirect.org/201...ansmitters/gbr
Gives the human name and the lineupID associated with that transmitter.

So, if it's possible for the end-user to supply a lineupID directly, that may be useful in non-postal code based lineups, like satellite. It also helps users in Europe, where we may not have lineups for that country per-se, but they are still in a ZZZ-based satellite lineup footprint, like Astra 19.2 or 28.2E.

[EnterNoEscape]

The JSON being returned is just

{"DVB":{"headend":"DVB-S","transport":"Satellite","location":"Global"}}

[rkulagow]

enternoescape - that just shows you how stale the code is. It's a feature that we implemented as a black-box (because I have no satellite equipment), and no one uses it. I've requested developers contact me in the Wiki documentation and no one has:

(https://github.com/SchedulesDirect/J...g-for-a-lineup)

If it's something that's actually useful and users other than pendragon will take advantage of it, then contact me at rkulagow@gmail.com and we'll work through providing useful information.

However, at this time, each satellite is provided as a separate lineup, and pendragon has already let me know that SageTV won't like that, so we may need to coordinate on creating either a new endpoint to glob all satellites in NA into a lineup, or SageTV would need to accommodate multiple lineups associated with one source. (The problem is what to do in fringe areas; if I setup a lineup that has all the "European" satellites, then what do I do about users in Eastern Europe that have reception for Europe and Asia? Etc)

Satellite users: this is your chance to weigh-in with the reality of FTA since I really have no point of reference.

If we need to move FTA satellite to another thread over at Schedules Direct then I'm fine with that as well.

[EnterNoEscape]

Originally Posted by rkulagow

In some areas of the world it's easier to just put in the lineupID if it's known. For example, because of how postal code coverage works in places like the UK, your "best" transmitter (as known by "you") may not be in your postal code for whatever reason. That's why we publish

https://json.schedulesdirect.org/20141201/available

Which you can see has a dvb-t REST endpoint available (currently only GBR, but we had AUS and NZL back when we were still experimenting with them)

https://json.schedulesdirect.org/201...ansmitters/gbr
Gives the human name and the lineupID associated with that transmitter.

So, if it's possible for the end-user to supply a lineupID directly, that may be useful in non-postal code based lineups, like satellite. It also helps users in Europe, where we may not have lineups for that country per-se, but they are still in a ZZZ-based satellite lineup footprint, like Astra 19.2 or 28.2E.

I thought I understood what "available" was mostly there for, but I did not know that it presents lineups that you cannot get any other way. I was of the impression that ZZZ was supposed to take care of all of those.

I've been trying to avoid implementing special cases because it just adds more paths to confuse users, but I see this situation requires it. I will have to add another screen so we can make it possible to add by lineup ID. I'm a little unenthusiastic about this because I don't think it should be necessary, but I will get the feature in there at some point.

[EnterNoEscape]

SageTV as we know only supports one lineup per capture device input (not per capture device). This assumption is made enough throughout the code that changing it to support a many to one relationship is unlikely to be a simple change. It would also entail some UI changes that for me at least would be even harder to deal with correctly than the Java code.

While I can think of ways to shim support in there so it thinks it's only dealing with one lineup when it actually is multiple lineups, I have my doubts that this approach will be reliable because there are more things that need to be considered than just adding the lineups.

[pendragonsound]

FTA schedules worked with the old, licensed Sage EPG code because I manually added custom channels and entered the appropriate Station IDs. I never configured a lineup for my FTA sources. The previous EPG server didn't require each Station ID to be in a lineup I declared in my Sage configuration. That has changed with SD.

The reason I'm reiterating is because there is no compelling reason for me to interact with these types of lineups in Sage. They simply need to be listed in my SD account. They don't have to be associated with Sage content sources. I would still have to manually enter each channel and Station ID as before.

Thus all I need is a means to add non-associated lineups to my SD account, without having to tie them to a Sage source. That could be done on a separate screen to keep the existing interface simple and less confusing for most users. This screen would only display the non-associated line-ups, a distinction known only to the Sage SD code. That way one couldn't accidentally manipulate lineups that were tied to content sources.

[EnterNoEscape]

Originally Posted by pendragonsound

Thus all I need is a means to add non-associated lineups to my SD account, without having to tie them to a Sage source. That could be done on a separate screen to keep the existing interface simple and less confusing for most users. This screen would only display the non-associated line-ups, a distinction known only to the Sage SD code. That way one couldn't accidentally manipulate lineups that were tied to content sources.

I can add this; it involves adding at least one more screen to the V7 STV and some additional code to deal with adding a lineup to your account by ID. For me, it's not a trivial change because it requires changes to the STV, but I will get it done eventually.

Edit: Anyone that has worked in the STV knows this, but a lot of the screens you see seem deceptively simple when you're interacting with them.

[pendragonsound]

Originally Posted by EnterNoEscape

I can add this; it involves adding at least one more screen to the V7 STV and some additional code to deal with adding a lineup to your account by ID. For me, it's not a trivial change because it requires changes to the STV, but I will get it done eventually.

Perhaps there is a straightforward solution that doesn't require any coding. I've just now been playing with mfdb-json, which is a command line grabber for Myth. But it can run standalone. There are a few quirks I'm still working out, but it appears I may be able to use this to configure (add/remove) additional lineups to my SD account. Will the Sage SD code balk if there are additional lineups for my account that have not been configured through Sage?

[EnterNoEscape]

Originally Posted by pendragonsound

Perhaps there is a straightforward solution that doesn't require any coding. I've just now been playing with mfdb-json, which is a command line grabber for Myth. But it can run standalone. There are a few quirks I'm still working out, but it appears I may be able to use this to configure (add/remove) additional lineups to my SD account. Will the Sage SD code balk if there are additional lineups for my account that have not been configured through Sage?

The SageTV code just reads back what's on your account. It doesn't care if it put it there or not. I'm going to add this feature either way since I'm sure we will see another instance in the future where it would be nice to have.

[pendragonsound]

This may not quite be the correct thread, but my issue is a follow-on to this. Several months ago when I converted to Schedules Direct, I ran into problems trying to pull EPGs for a number of FTA channels. This had worked fine for me in Sage 7. Since then, SD and I have been working on an experimental lineup for North American FTA satellites under the "country" code ZZZ. Hence this thread.

This is a work in progress, and for the moment there are only a couple of orbital locations in this lineup (58W and 125W). Yesterday I tried to add the "ZZZ-FTA.NORTHAMERICA-DEFAULT" lineup to my SD account through Sage, but when I entered the country code ZZZ, nothing was listed. Thus I couldn't select the lineup. However the lineup is shown at:

https://json.schedulesdirect.org/201...vailable/dvb-s

Instead I used the mfdb-json command line utility to add the lineup to my SD account. I next went to Sage to attach this lineup to a couple of FTA tuners. The lineup was now listed as one of the four associated with my SD account, and I was able to select it as the EPG source for both FTA tuners. However the Sage client got stuck saying it was in the process of downloading the channel list, and that never completed.

Because the channel list in reality has nothing to do with the what and how my FTA tuners access the channels, this wasn't important to me. I've always entered this manually or import an existing list using a Sage plugin. I did the latter and adjusted several of the stationIDs to match what is in the FTA lineup. Since then I have not seen any of the FTA channels get their EPG updated. I did notice the SD code in Sage appears to be having problems with the FTA lineup. The following is repeated fairly often in the logs:

Wed 10/18 14:00:23.278 [EPG@12054d8] EPG attempting to expand Local Over the Air Broadcast - 80111 (sdepg)
Wed 10/18 14:00:23.280 [EPG@12054d8] expand called on Local Over the Air Broadcast - 80111 (sdepg) at Wed 10/18 14:00:23.280 expandedUntil=Wed 10/18 20:47:54.051 scannedUntil=Sun 8/17 0:12:55.807
Wed 10/18 14:00:23.280 [EPG@12054d8] EPG attempting to expand DISH Denver - Satellite - Denver (sdepg)
Wed 10/18 14:00:23.281 [EPG@12054d8] expand called on DISH Denver - Satellite - Denver (sdepg) at Wed 10/18 14:00:23.281 expandedUntil=Wed 10/18 17:10:51.448 scannedUntil=Sun 8/17 0:12:55.807
Wed 10/18 14:00:23.281 [EPG@12054d8] EPG attempting to expand Shaw Direct (Advanced) - Satellite - Canada (West) (sdepg)
Wed 10/18 14:00:23.282 [EPG@12054d8] expand called on Shaw Direct (Advanced) - Satellite - Canada (West) (sdepg) at Wed 10/18 14:00:23.281 expandedUntil=Wed 10/18 22:30:17.775 scannedUntil=Sun 8/17 0:12:55.807
Wed 10/18 14:00:23.282 [EPG@12054d8] WARNING: Aborting properties file save!
Wed 10/18 14:00:23.283 [EPG@12054d8] sage.epg.sd.SDRipper@103172b needs an update in 6:47:30
Wed 10/18 14:00:23.283 [EPG@12054d8] sage.epg.sd.SDRipper@e3f539 needs an update in 3:10:28
Wed 10/18 14:00:23.283 [EPG@12054d8] sage.epg.sd.SDRipper@55a5a needs an update in 8:29:54
Wed 10/18 14:00:23.283 [EPG@12054d8] sage.epg.sd.SDRipper@1a35fcb needs an update in 0:00:00
Wed 10/18 14:00:23.283 [EPG@12054d8] EPG needs an update in 0 minutes
Wed 10/18 14:00:23.283 [EPG@12054d8] EPG PRIORITY EXPANSION attempting to expand FTA - North America - North America (sdepg)
Wed 10/18 14:00:23.283 [EPG@12054d8] expand called on FTA - North America - North America (sdepg) at Wed 10/18 14:00:23.283 expandedUntil=Wed 10/18 13:59:21.664 scannedUntil=Sun 8/17 0:12:55.807
Wed 10/18 14:00:24.475 [EPG@12054d8] EPG Expanding FTA - North America - North America (sdepg) at Wed 10/18 14:00:24.475
Wed 10/18 14:00:24.874 [EPG@12054d8] SDEPG Exception thrown: java.lang.NumberFormatException: For input string: "11916500000"
Wed 10/18 14:00:24.875 [EPG@12054d8] sage.epg.sd.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "11916500000"
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.TypeAdapters$7.read(TypeAdapters.java:245)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.TypeAdapters$7.read(TypeAdapters.java:235)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:130)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:221)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:72)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.Gson.fromJson(Gson.java:889)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.Gson.fromJson(Gson.java:954)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.TreeTypeAdapter$GsonContextImpl.deserialize(TreeTypeAdapter.java:163)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.json.map.SDLineupMapDeserializer.deserialize(SDLineupMapDeserializer.java:58)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.json.map.SDLineupMapDeserializer.deserialize(SDLineupMapDeserializer.java:30)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.TreeTypeAdapter.read(TreeTypeAdapter.java:70)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.Gson.fromJson(Gson.java:889)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.gson.Gson.fromJson(Gson.java:827)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.SDSession.getAuthJson(SDSession.java:689)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.SDSession.getLineup(SDSession.java:837)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.epg.sd.SDRipper.extractGuide(SDRipper.java:1236)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.EPGDataSource.expand(EPGDataSource.java:709)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at sage.EPG.run(EPG.java:742)
Wed 10/18 14:00:24.876 [EPG@12054d8] 	at java.lang.Thread.run(Unknown Source)
Wed 10/18 14:00:24.877 [EPG@12054d8] Caused by: java.lang.NumberFormatException: For input string: "11916500000"
Wed 10/18 14:00:24.877 [EPG@12054d8] 	at java.lang.NumberFormatException.forInputString(Unknown Source)
Wed 10/18 14:00:24.877 [EPG@12054d8] 	at java.lang.Integer.parseInt(Unknown Source)
Wed 10/18 14:00:24.877 [EPG@12054d8] 	at java.lang.Integer.parseInt(Unknown Source)
Wed 10/18 14:00:24.877 [EPG@12054d8] 	at sage.epg.sd.gson.JsonPrimitive.getAsInt(JsonPrimitive.java:260)
Wed 10/18 14:00:24.877 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.JsonTreeReader.nextInt(JsonTreeReader.java:244)
Wed 10/18 14:00:24.877 [EPG@12054d8] 	at sage.epg.sd.gson.internal.bind.TypeAdapters$7.read(TypeAdapters.java:243)
Wed 10/18 14:00:24.877 [EPG@12054d8] 	... 19 more
Wed 10/18 14:00:24.877 [EPG@12054d8] ERROR Updating EPG Source FTA - North America - North America (sdepg)
Wed 10/18 14:00:24.877 [EPG@12054d8] ErrorText:expand called on FTA - North America - North America (sdepg) at Wed 10/18 14:00:23.283 expandedUntil=Wed 10/18 13:59:21.664 scannedUntil=Sun 8/17 0:12:55.807
EPG Expanding FTA - North America - North America (sdepg) at Wed 10/18 14:00:24.480

I'm hoping you will be able to help.

[EnterNoEscape]

The problem is that 11916500000 > 2147483648, so it can't be a signed 32-bit integer. I have a feeling that's coming in as a station ID which SageTV will not be able to handle correctly if it needs to be a 64-bit signed integer (long) and the workaround we're using for Australia and New Zealand would also be broken if we need to use all 31 positive bits.

My free time is very limited right now. I used to check the forums daily, now I feel like it's a lot of effort to just check once a week. As time permits, I will try to see what we can do to make this work.

[pendragonsound]

Originally Posted by EnterNoEscape

The problem is that 11916500000 > 2147483648, so it can't be a signed 32-bit integer. I have a feeling that's coming in as a station ID which SageTV will not be able to handle correctly if it needs to be a 64-bit signed integer (long) and the workaround we're using for Australia and New Zealand would also be broken if we need to use all 31 positive bits.

I just looked at the raw data I sent Schedules Direct, and 11916500000 is the frequency (in Hz) of the first channel in the North American lineup. This is a slightly massaged version of what I sent, where I've updated the fields they inserted:

 {"stationID":"43025","frequencyHz":"11916500000","polarization":"V","symbolRateHz":"2398000","deliverySystem":"DVB-S2","modulationSystem":"8PSK","fec":"2\/3","serviceID":"1","matchType":"providerCallsign","providerCallsign":"KUSMDT"}

As far as I'm concerned, the extra fields can be ignored.

[EnterNoEscape]

Originally Posted by pendragonsound

I just looked at the raw data I sent Schedules Direct, and 11916500000 is the frequency (in Hz) of the first channel in the North American lineup. This is a slightly massaged version of what I sent, where I've updated the fields they inserted:

 {"stationID":"43025","frequencyHz":"11916500000","polarization":"V","symbolRateHz":"2398000","deliverySystem":"DVB-S2","modulationSystem":"8PSK","fec":"2\/3","serviceID":"1","matchType":"providerCallsign","providerCallsign":"KUSMDT"}

As far as I'm concerned, the extra fields can be ignored.

I had not had the time to look at anything more than what you posted, so I was just guessing that was a station ID.