OpenDCT: An open source digital cable tuner.

[crawfofd]

I downloaded the the new alpha (4.4) and made all the changes to the properties file. Did not fix the problem. I have attached another log file for your review.

[EnterNoEscape]

I think I just noticed something critical after taking a closer look at what's being written into the buffer. You're maxing out the internal UDP packet receive buffer. Somehow you're receiving UDP packets larger than 1500 bytes. Even with jumbo packets enabled, that shouldn't increase the packet sizes from the InfiniTV devices. The normal MTU is 1500 or less. I'll have a new build for you later today.

Update: To clarify, I know you can send larger UDP packets, but normally RTP will not do that. It generally creates packets that are 1328 bytes in size (OpenDCT uses a 1500 byte buffer) because that's the most 188 byte TS packets plus the 12 byte RTP header plus overhead you can fit into the 1500 byte packet.

[crawfofd]

Getting closer, I can now display the ceton output and it looks pretty good. Still getting quite a few errors in the log. It also look likes its affecting both the Ceton and the Prime, although with this release it more the Prime then the Ceton. I have two files for you in one zip. One was done while the system was running while the other was the end where I stopped the service.
Hope this helps, Its definately getting better. And if I havent said it enough Thanks for all the hard work.

[Telecore]

I performed a test from work via Remote Desktop, so I really can't speak for the quality of the picture, however, the latest alpha does appear to be rendering video. Looks like good progress. My log files are attached - there appear to be some warnings in there. I will see how video appears on an HD300 when I get home later. Thanks!

[EnterNoEscape]

Wow, those packets must be massive. I'm switching over to a dynamic design that will just allocate more RAM to receiving with a hard limit of 5MB when allocation issues are detected and then retains the peak value in properties so the next use won't need to figure it out all over again. I would hate to allocate so much RAM unless it's actually needed, so if this is how some of the carriers are going to operate, I'll need to create a compromise and I think this will be good enough.

[crawfofd]

What ever you say EnterNoEscape, your way over my head. I can test it but have no idea what your doing. I did see by the log that there wasnt enough memory. Is there anything on my end I can do? Remember if its technical you will have to explain.

Note. The output looks pretty good, but I only watched for a small time. I will reconfigure it and check the output for a longer period. I dont know if this matters but I do have 9 tuners, 6 Ceton and 3 prime.

[crawfofd]

Another note. Before the last build the Primes always seem to have priority, but with the last build it seems the Ceton is getting higher priority. It might just be my imagination.

[EnterNoEscape]

Originally Posted by crawfofd

What ever you say EnterNoEscape, your way over my head. I can test it but have no idea what your doing. I did see by the log that there wasnt enough memory. Is there anything on my end I can do? Remember if its technical you will have to explain.

Note. The output looks pretty good, but I only watched for a small time. I will reconfigure it and check the output for a longer period. I don't know if this matters but I do have 9 tuners, 6 Ceton and 3 prime.

The tuner configurations shouldn't really matter. My load testing is with 11 tuners in a VM and it gets by without any crazy issues. The error message that you saw doesn't exactly mean we ran out of allocated memory per say, it just means that the returned packet exactly matched the size of the buffer which can mean that it's not big enough. Unfortunately in Java NIO will not actually tell you how big the packet was, you just need to infer that when it fills the buffer, it's possible it dropped part of the packet. I think I hit the exact value needed if you're saying it's working now. Also, it should only take less than 5 seconds of watching to completely cycle through the circular buffer too. So you likely make it through at least one cycle there without any issues which is also important since I made some significant changes to how that works. I didn't see anything of interest from FFmpeg in your logs, so you're probably good. I'm still going to make the buffer dynamic to counter-measure this type of issue in the future. I guess there must be something about the stream coming into the tuner that's causing it to not be able to break it up so it's just streaming the data in massive chunks. I hope that wasn't too technical. I'll post another version for you both just to clean up the logs because it looks like you're going to see that warning message a lot otherwise.

Originally Posted by crawfofd

Another note. Before the last build the Primes always seem to have priority, but with the last build it seems the Ceton is getting higher priority. It might just be my imagination.

Tuner selection as far as I can tell in SageTV is mostly influenced by merit by the scheduler, so I'm going to say it's your imagination unless you have played around with the merit properties. The only other possibility would be that pooling is enabled. Pooling is a new feature in the builds you're testing, but it's not enabled by default as you can see when you tell SageTV to use a particular tuner, it's using the intended tuner. With pooling enabled you would actually see in the logs that OpenDCT selected the a tuner for you and mapped it to the tuner SageTV thinks it's using. The advantage is mostly for users of the Prime tuners, so if for example you're watching something streaming from the tuner, it will try to use a different tuner and only steal that tuner away from you if it doesn't see any other options. This selection process is also weighted by the merit value set in opendct.properties. Anyway, it won't interest you unless you want to be able to more conveniently use your Prime for anything other than SageTV.

[crawfofd]

Thanks for your explaination. Will the update that you send us stop the warnings? The log file fills and cycles through pretty quickly. Its doing a lot of writes.

[EnterNoEscape]

Originally Posted by crawfofd

Thanks for your explaination. Will the update that you send us stop the warnings? The log file fills and cycles through pretty quickly. Its doing a lot of writes.

Yes. Those will go away. That would be quite a problem for you to have those constantly logging.

[crawfofd]

Thanks, will download and run in a few.

[Telecore]

Just got back home - I watched a channel using the previous alpha and it actually looked very good on my HD300 - I didn't see any errors, but I didn't watch very long. I saw your PM and installed the latest 0.4.6 alpha and the log files seem a lot cleaner. They are attached. I have installed the 64 bit version this time and will try using it exclusively for watching some shows later this week. I'll let you know if I see any issues.

I am doing all of this on my 2nd/experimental SageTV system which I will eventually make the primary one - I am happy to get OpenDCT working with the latest jars/V9 and exes. Thanks a lot for your hard work. Now I can only hope that when Verizon changes to Frontier in a few months that they don't turn on all the CCI flags.

[EnterNoEscape]

How's that for irony. I have FiOS at my location too. Now I'm really confused about why the packets are so big. I was thinking it must be a provider issue since I have the same hardware and in crawfofd's case we are using the same everything, but I guess this will just be one of those mysteries. I feel much better now than this appears to have been addressed.

I saw the one warning I expecting before the size was automatically bumped up to 32k which is just a little bigger than the largest UDP packet that should be possible and didn't see it increase in size again, so it is working as expected. The FFmpeg info looks very normal too. I think you'll be as good as anyone else.

[crawfofd]

Okay, was able to do testing on 4.6a and it seems to be working pretty good. I have 9 tuners. Currently watching 4 shows (1 Local, 1 client, 1 hd300, 1 miniclient on android). Also recording 4 shows! Pretty good cpu @ 25%.

Thats the good news, this is what I have noticed so far. I am getting 'Halt detected errors' not a lot but seems consistant and it is only on the Hdhomerun Prime. (the prime works fine on my ver 7 system). Also when I view the video source the prime does not show up as a network encoder, is this normal?

All and all great job for a alpha/beta. For the last year I have been trying to get Jriver media center to work for me like sagetv, but it has always been a up hill battle. With this and all the other projects now on this forum I can drop Jriver.

I have attached the log files with the errors. Thanks again!

[Telecore]

I used to run two HDHR Primes. I am pretty sure that they use the normal "hardware driver" approach (No, actually I had to use SageDCT, so it was a network encoder) and are not network encoders. They worked pretty well for me, but I always had occasional glitches, despite a lot of efforts to optimize my network situation. I tried a lot of things, installing an Intel NIC and changing a lot of settings, but could never get rid of 1 or 2 very minor glitches while watching a show. That is why I now run the Ceton PCIe - everything works better "in the box".

But your "halt detected" should not be happening unless you are dropping packets or if the CPU is overloaded. You should try and optimize your network card/chip settings.

[EnterNoEscape]

Are you sure you're using the Prime via OpenDCT? It was detected in all of the logs, but it wasn't actually used. I suspect it's not configured on the SageTV side based on your later comment about it not appearing under video sources. Double-check that it reads DCT-HDHomeRun Prime Tuner 1321904D-0. I can see that SageTV did perform a discovery, so if it's not available to be added, that might be part of the problem and could also validate your comment about the InfiniTV devices seeming to have preference over the Prime suddenly. I suspect something isn't right with how it's loaded on the SageTV side. Also if somehow it's not coming in via discovery, that would be a much bigger issue and odd that it's not generating any errors in the OpenDCT logs. If you still can't figure it out, please attach your Sage.properties file so I can see if I can tell you what is probably happening.

Something of note for everyone: As far as I know it's only recent (mid-2015) that we have wide-spread use of the network encoder discovery feature in SageTV. There is a side-effect that will be very annoying for some people. If you run multiple copies of any network encoder that support this feature, they will all respond to the discovery request and SageTV will import them all along with potential duplicates of the same device because the IP addresses of the network encoders differ. If you are one of these people you will need to carefully look at the IP address and make sure it's the right network encoder at the expected address. Right now, the only thing you can do in OpenDCT to prevent this is tell OpenDCT to not load the unused capture device(s) in opendct.properties and that will prevent it from showing up on discovery for that particular running copy of OpenDCT.

I know how you feel. If I knew SageTV would have made this kind of a comeback I probably would have started working on this sooner. I was hopeful that Google would open source it, but without knowing, it was hard to tell if it would be worth my while when SageDCT was working reasonably well for me. Even after the open source announcement I was a little skeptical about about the community reception and waited to see if things really picked up as people were anticipating. Then I finally got sick of not being able to use my InfiniTV 4 for anything without a CableCARD, started playing with the Cling library and a project with no name was born.

[crawfofd]

Telecore, Thanks for your info. Could you give me a little more info on the network card optimize settings??

[crawfofd]

EnterNoEscape, As always you were right on the money. When I entered the Prime as a video source there were multiple selections with the same tuner but different IPs. Ofcourse I selected the one that didnt have DCT but it loaded and did work just gave the errors. I picked the IP that was the same as the computer, but had to pick the 127.0.0.1 which I thought wouldnt have worked. Its been running recording 7 streams and viewing on 1 client and 1 hd300 with no (0) errors.

Awesome!

I have converted everything to ver 9 OS and am now looking to go to Gemstone.

If you need any testing at all in the future I volunteer.

Thanks again.

[Telecore]

Originally Posted by crawfofd

Telecore, Thanks for your info. Could you give me a little more info on the network card optimize settings??

See the attached file

[HelenWeathers]

I've run into an issue with OpenDCT. Not sure if you're equipped to test or not. Hope someone else may be able to try this also.

This only happens when recordings on my OTA tuners start while recordings on DCT tuners are in progress. This is with Sage 7 and win 7/32 with 4 OTA tuners and OpenDCT w HDHR Prine CC tuner. OpenDCT is running on the same machine (my Sage server). OpenDCT ver = OpenDCT_0.3.9_x86.

To test I set up 2 recordings on DCT tuners to start at ie 2PM and 4 recordings on OTA tuners starting at 2:05, 2:06, 2:07 and 2:08PM. All 6 recordings stopped at 2:15PM.

All 4 OTA recordings were glitch free but both DCT tuner recordings exibited 6-8 second glitches at 2:05, 2:06, 2:07 and 2:08 when the 4 OTA tuners kicked in.

I havn't upgraded to a newer ver of OpenDCT because this ver seemed to be verry solid until I realised that I was getting some of these glitches in my DCT recordings. In reading this thread daily it seemed that most of the mods made lately have been for special cases.

Is there a later stable release you want me to try? Any logging I should turn on (and if so where to turn it on)?

I'm not desperate for a solution as most of my recordings get watched and deleted so a glitch here and there is not currently a big deal.