Debugging Hung SageTV in Docker

[broconne]

I am using the stuckless/sagetv-server-java8:latest docker container for SageTV. I have two HDHR tuners.

I have an issue where SageTV runs for maybe a couple of days and then hangs. I am running this on an Ubuntu server. I see no exceptions in the logs, the most recent "MemStats" entries look ok. SageTV just stops logging.

Are there any tips to figure out where to look as to why SageTV is just hanging in docker?

Below are the last few log lines I see before it stops.

Mon 8/6 0:12:29.450 [Ministry@a24d060] Ministry is waiting for 180 sec
Mon 8/6 0:12:32.981 [ThreadMonitor@70aeac6] Top 10 Cumulative CPU(sec) Timer-0:56 AsyncPropSaver:32 1363498639@qtp-468237354-8_-_Acceptor1_SelectChannelConnector@0.0.0.0:8080:7 902980376@qtp-468237354-9_-_Acceptor0_SelectChannelConnector@0.0.0.0:8080:7 EPG:6 HotplugStorage:5 main:4 Seeker:3 MiniBootp:2 MiniMVP:2
Mon 8/6 0:12:44.212 [ThreadMonitor@70aeac6] Top 10 %CPU (last 300s)
Mon 8/6 0:15:04.998 [Fork-XJXYO-750@734eae96] Updating weather
Mon 8/6 0:15:20.782 [Fork-XJXYO-750@68efee82] Updating weather
Mon 8/6 0:15:31.046 [Seeker@f268a27] Seeker awoken
Mon 8/6 0:15:34.857 [Ministry@a24d060] Ministry is waiting for 180 sec
Mon 8/6 0:15:44.277 [Seeker@f268a27] MemStats: Used=253MB Total=763MB Max=805MB
Mon 8/6 0:15:54.209 [Seeker@f268a27] MARK 1 currRecord=null enc=HDHomeRun 101773dc Tuner 1 clients=[] ir=false
Mon 8/6 0:16:01.751 [Seeker@f268a27] Seeker in AUTOMATIC mode nextRecord=null nextTTA=9223372036854775807
Mon 8/6 0:16:10.261 [Seeker@f268a27] newRecord=null
Mon 8/6 0:16:18.578 [Seeker@f268a27] NOTHING TO RECORD FOR NOW...
Mon 8/6 0:16:24.786 [Seeker@f268a27] MARK 1 currRecord=null enc=HDHomeRun 101773dc Tuner 0 clients=[] ir=false
Mon 8/6 0:16:32.963 [Seeker@f268a27] Seeker in AUTOMATIC mode nextRecord=null nextTTA=9223372036854775807
Mon 8/6 0:16:40.664 [Seeker@f268a27] newRecord=null
Mon 8/6 0:16:47.564 [Seeker@f268a27] NOTHING TO RECORD FOR NOW...
Mon 8/6 0:16:56.420 [Seeker@f268a27] MARK 1 currRecord=null enc=HDHomeRun 1017a350 Tuner 0 clients=[] ir=false
Mon 8/6 0:17:01.837 [Seeker@f268a27] Seeker in AUTOMATIC mode nextRecord=null nextTTA=9223372036854775807
Mon 8/6 0:17:07.761 [Seeker@f268a27] newRecord=null

[stuckless]

Do you know if the java process is still running? Do you see any java crash logs? Maybe jvm is just crashing?

[broconne]

Thanks for reponing Stuckless.

Good question, if I attach to the container, I don't see the java process running.

I don't see any java crash logs. I searched for anything with "java" and "core" in it and came up empty.

[stuckless]

Originally Posted by broconne

Thanks for reponing Stuckless.

Good question, if I attach to the container, I don't see the java process running.

I don't see any java crash logs. I searched for anything with "java" and "core" in it and came up empty.

It's odd that there is no indication as to why sagetv is stopping. typically is java crashes, it write a javacore file in the home directory of the java process (so, /opt/sagetv/server). Typically if another plugin forces sagetv to stop (ie, tcp server was doing this), you'd see some logging about sagetv exiting.

I'm using the docker container on unRAID and I go months without rebooting/restarting, etc, and I've never had it just die

[broconne]

Perhaps the docker daemon is killing it.. My logs don't go that far back. I will restart and keep an eye out and then see if I find anything in the docker logs.

[broconne]

During troubleshooting I found some errors in the docker logs. Following the thread of the errors, I was on and old version of docker (1.13). Updated docker, rebooted, so far so good.