HDPVR Halts in unRAID

[willieb]

I have spent the weekend investigating an HDPVR issue that appeared when I switched to unRAID that I never had under Windows.

This issue has been discussed a little here and here, but I thought I should start a new thread in case others switch to unRAID and encounter this issue.

I have run 2 HDPVRs connected via Component + SPDIF to my FiOS boxes (QIP-7100s) since the HDPVR was released. They have been VERY stable since about 2013 when Hauppauge released the most recent driver.

After running on unRAID for about 2 days, I started getting "halts" in recordings. Often multiple times in one recording. Each time SageTV would be able to recover and restart the HDPVR. It would split the recording off into a new file and keep going after missing 60 seconds or so of the show/movie. It would keep going until the next "halt" and then do the same again. This resulted on 4 or 5 segments to the recording. This did not occur on ALL HDPVR recordings, which made it tough to diagnose.

After about a bazillion tests, I think I have figured it out. The halts are caused by the audio switching from 5.1 back to stereo (or maybe the reverse).

The thing that helped me figure it out was I realized a lot of my halts were happening on FX shows (The Americans). They were not happening on Premium channels when recording movies. I have to record FX using my HDPVR because they are flagged Copy Once on FiOS. From here I started testing and looking at where the halts were happening. I recorded the same movies twice (once in the morning and again when it was showed again in the afternoon). Generally, the halts happened in the same places in the movies. On top of that, I noticed that the halts always happened during or right after commercials.

I had my boxes locked to 1080i, so I was pretty sure the halts weren't caused by resolution changes. Just to be sure, I locked them both to 720p and that had no effect on the halts.

That led me to believe it had to be audio. I switched one box over to use the Stereo RCA inputs.

I then recorded the same movie that had given me halts before on FX and it made it all the way through without a single halt. I have since recorded several shows, again with no halts. So I am confident that is the issue. Interestingly, I think this was an issue many years ago on Windows, and I wonder if it could be solved by compiling a different driver into the kernel (but it isn't something I can do without re-learning a ton of Linux-fu I forgot about a decade ago).

Losing 5.1 on my HDPVRs is not good for me. I use it with my surround sound system and it adds to my enjoyment of shows/movies.

For now, I have set a new lineup on the HDPVR that I have switched to stereo and that is the only box where I have Fox Copy Once channels enabled. I figure I will almost never be recording form more than one at a time. That way, they will always record on that HDPVR. I also set it at lowest merit. My other HDPVR still is set to use 5.1 over SPDIF. I am hoping since it has a higher merit and gets fewer channels, SageTV will use it whenever I want to record HBO, Starz, etc. so I can get 5.1 sound.

I would be interested to hear other peoples' experience. I know Stuckless has said he does not have this issue, but I wonder if you use SPDIF or if having a different cable provider might be the issue. I have tried to be as detailed as possible here so others can compare their situation to mine.

I hope this is helpful.

[BobPhoenix]

Wish I had time to test my problems as I suspect some/most of them are probably the same as yours. I wouldn't want to switch to Stereo from 5.1 either. Maybe in a couple of weeks I can test possible causes. For now my temporary solution (as you know) - that mostly works - is to use the HDPVRs in a VM with the 1.5.7 windows drivers. But I really have no desire to make that a permanent solution if I can find a native one. So I will be following your efforts to a solution and if I get the opportunity trying some of my own. If I figure out how to fix my system I will post back here with what worked for me.

Note if you want me to remove this post I will.

[willieb]

Originally Posted by BobPhoenix

Wish I had time to test my problems as I suspect some/most of them are probably the same as yours. I wouldn't want to switch to Stereo from 5.1 either. Maybe in a couple of weeks I can test possible causes. For now my temporary solution (as you know) - that mostly works - is to use the HDPVRs in a VM with the 1.5.7 windows drivers. But I really have no desire to make that a permanent solution if I can find a native one. So I will be following your efforts to a solution and if I get the opportunity trying some of my own. If I figure out how to fix my system I will post back here with what worked for me.

Note if you want me to remove this post I will.

Of course not. Glad to have input. I considered the VM myself and might go that route in the interim, but for now I will see if I can live with the approach of bifurcation of my HDPVR usage. I would be interested in hearing from the Linux masters as to how hard it would be to compile a different driver into the unRAID kernel. My guess is that it is not trivial.

[wayner]

I am one of those that has issues with the first minute or two of recordings but almost always things are good after that. I will try to keep an eye to see if this is an issue, or maybe even try to see if I can temporarily force recordings to stereo to see if that makes the issue go away.

But I am guessing that my issue is similar but a bit different.

[stuckless]

And that explains why I don't have an issue... I have always used stereo output. So, would this issue be in SageTV or in the HDPVR driver or somewhere else?

[wayner]

I also occasionally get these when recording from an HDHR OTA tuner in addition to an HD-PVR. Has anyone else observed that problem as well? That's one of the things making me think it was related more to drive spin-up rather than the actual tuner.

[BobPhoenix]

Originally Posted by wayner

I also occasionally get these when recording from an HDHR OTA tuner in addition to an HD-PVR. Has anyone else observed that problem as well? That's one of the things making me think it was related more to drive spin-up rather than the actual tuner.

I get them on OTA as well but it is always a signal problem for me. Not sure why the station signal will be going along at 100% and then drop below 33% for a few seconds but that is what happens to me on OTA (and even QAM) signals. So I don't think (for me at least) there are any problems other than with the HDPVR.

I've been recording in 5.1 since the 1.5.7 HDPVR drivers came out for Windows. Will not go back to the pre 1.5.7 stereo recordings.

I think the linux drivers are the problem - based on my experience with the Windows drivers anyway. I tried to update to newer drivers for Windows in the past and it always f**ked up 5.1 sound and I had to roll back. So I just stopped trying several years ago and stuck with the 1.5.7 Windows drivers.

[wayner]

It is possible that it is a signal problem for me as well but I am still using my Windows based V7 system and both it and my unRAID V9 system recorded golf yesterday afternoon. I had two errors occur while recording PGA Tour golf from 2:30-6:30 yesterday on the unRAID server but no errors on the Win server. They were using different tuners so there could be different signal levels but it is kind of strange.

The other thing that is weird is that with OTA these recording errors almost always happen near the top of the hour. For example the golf errors occurred at 5:01:35 and 6:02:50 yesterday. Saturday night during an OTA hockey game an error occurred at 9:01:30, and for Saturday afternoon golf there was an error at 5:02:59 and 4:01:28. The previous week I had an error during golf at 4:02:59 and 6:01:43.

The errors that occur about three minutes after the error typically have occurred five or six times which I can see by a 5X or 6X on the System Messages screen in the web UI. The errors that are within a minute or two of the hour are only 1X to 3X errors.

Is there anything special that could cause an error at the top of the hour in either Sage or unRAID?

[willieb]

Originally Posted by stuckless

And that explains why I don't have an issue... I have always used stereo output. So, would this issue be in SageTV or in the HDPVR driver or somewhere else?

I think it is in the HDPVR driver. It looks like the HDPVR just stops receiving data (or at least gets confused enough by the switch that it decides it is getting garbage and stops sending data to Sage). My understanding of a halt is that Sage stops receiving data from the encoder for a specified period of time (defined in properties file -- usually 60 seconds). I would be interested to see if an earlier driver would fix this, but for all I know that would require moving mountains.

Originally Posted by BobPhoenix

I get them on OTA as well but it is always a signal problem for me. Not sure why the station signal will be going along at 100% and then drop below 33% for a few seconds but that is what happens to me on OTA (and even QAM) signals. So I don't think (for me at least) there are any problems other than with the HDPVR.

I had really erratic signals for a while on my OTA. It would fluctuate from 98% to 22% and back in a matter of seconds. I finally figured out this is mutipath interference and you just have to find a place for your antenna that minimizes it. Which is like trying to find your way in the dark. But then I found this awesome app in the iphone app store called Signal GH. It is worth its weight in gold (assuming apps have weight). I think it was $4 or 5 bucks. It gives you realtime feedback on your signal strength AND quality for every tuner coming out of your HDHR. You can tune to different channels and see strengths from different towers. It also has a map which shows you the directions of all of your towers.

Originally, I spent days in my attic trying to get a good signal. Whenever I thought I had it licked, I would go back downstairs and 20 minutes later my signal would be dropping out again. After I got this app, it took me 30 minutes of just walking around in my attic with the antenna and holding it up and watching the signal meter on my phone. I found a rock solid spot all the way at the opposite end of my house from where I had been trying. I've had no dropouts since.

[EnterNoEscape]

Originally Posted by willieb

I think it is in the HDPVR driver. It looks like the HDPVR just stops receiving data (or at least gets confused enough by the switch that it decides it is getting garbage and stops sending data to Sage). My understanding of a halt is that Sage stops receiving data from the encoder for a specified period of time (defined in properties file -- usually 60 seconds). I would be interested to see if an earlier driver would fix this, but for all I know that would require moving mountains.

I don't think the driver is as much interest as the firmware when it comes to the HD-PVR. MythTV is a good source of information about the HD-PVR behavior in Linux. Are you on the 0x1e firmware?

[wayner]

I didn't even know there was upgradeable firmware in the HD-PVR. Anyone know how to tell which firmware version you have?

[BobPhoenix]

Originally Posted by wayner

I didn't even know there was upgradeable firmware in the HD-PVR. Anyone know how to tell which firmware version you have?

Didn't know either would like to know myself.

[willieb]

Originally Posted by EnterNoEscape

I don't think the driver is as much interest as the firmware when it comes to the HD-PVR. MythTV is a good source of information about the HD-PVR behavior in Linux. Are you on the 0x1e firmware?

Yes, I am on 0x1e dated March 7, 2012. Guys, if you use PuTTY and type "dmesg | grep hdpvr" at the prompt, it should report to you what firmware you are on. (And I learned that this weekend from the MythTV forums - told you guys I spent the whole damned weekend on this).

Upgrading firmware requires doing it on a Windows machine. When you update the Windows drivers, it would update the Firmware, too.

[wayner]

I watched a recording to start to see how this evolves. The HD-PVR flashes on and tries to record, but after a second or two it shuts off. Then, about 74 seconds later it tries again and is successful.

I wonder if playing with a parameter like the delay_to_wait_after_tuning parameter would make a difference. I have mine set on 4000 which must be the default as I did not change it. What are other folks using?

[BobPhoenix]

I have 12000 on my Colossus on another server but I haven't increased my HDPVRs yet. Still at 4000.

[wayner]

I really wish you could change this in the UI rather than having to shut down the server and edit the Sage.properties file. I don't like having to kick everyone off the server to make a minor change like this and it isn't something that should require a server restart.

[wayner]

It is a small sample size but after a couple of recordings it looks like moving to a delay of 8 seconds is working at solving this problem.

[willieb]

Originally Posted by wayner

It is a small sample size but after a couple of recordings it looks like moving to a delay of 8 seconds is working at solving this problem.

My issue is never the initial tuning. I tune properly at the beginning. My issue as ALWAYS right after commercials. The audio changes back to stereo during the commercial and then back to 5.1 when the show starts. One of the two of those kills it.

[dstanley]

Originally Posted by wayner

It is a small sample size but after a couple of recordings it looks like moving to a delay of 8 seconds is working at solving this problem.

I have found over the years that 8 seconds was the best setting for me as well with the hdpvr and my stb's. It needs to have the stb channel change done and settled before the hdpvr kicks in for it to begin recording without a glitch. It seems the audio input is the most problematic for the hdpvr's. I have one stb that cannot seem to output a good of enough optical signal to record properly. Changing to analog helped but resulted in missed segments (skips) as the hdpvr struggles with the audio output. I ended up removing this stb (my oldest) out of my recording system.

[wayner]

Originally Posted by willieb

My issue is never the initial tuning. I tune properly at the beginning. My issue as ALWAYS right after commercials. The audio changes back to stereo during the commercial and then back to 5.1 when the show starts. One of the two of those kills it.

It could be a very similar issue though - the audio takes several seconds to kick in and the HD-PVR takes a few seconds to lock onto it. When it doesn't lock onto it in time it throws an error. For me, and for some others, this only happens on the initial sync up when it starts recording. In your instance it happens when the audio format changes when there is a commercial.