Plugin: SJQv4

[wayner]

Something weird happened - perhaps this is related to my lobs.db filling my hard drive.

When I look at the tasks for my main client (which is actually my server) there are only 8, there should be more, and they are all the same taskname "PROCKIDS". What could have caused this?

If I pull one of my WHS backups from a week or so ago are the tasks stored somewhere that I can pull some files and restore to the current server?

[Slugger]

What's missing? Tasks that have been completed? Or new tasks aren't being queued? You can't pull files from a backup to "replace" missing data. You could dump the database as it stands at the time of your backup then restore the db into the current db - maybe. I guess it depends on what exactly is "missing".

[wayner]

Not the jobs but the task definitions are gone, except for the same task that is repeated 8 times. For example my task "comskip_sd" no longer exists. So when I go into the UI and look at "Supported Tasks" there are only 8 and they are all identical.

[Slugger]

Originally Posted by wayner

Not the jobs but the task definitions are gone, except for the same task that is repeated 8 times. For example my task "comskip_sd" no longer exists. So when I go into the UI and look at "Supported Tasks" there are only 8 and they are all identical.

Hmm... that you might be able to restore. Those definitions are generated in the db from the client properties file.

/sagetv/plugins/sjq-agent/conf/sjqagent.properties

Check that out and see if it's corrupted - check your backups to see if a better version exists.

[wayner]

The sjqagent.properties file looks fine, it seems to have all of the properties for all of the tasks but it is somewhat hard to tell since the sorting is strange in that the properties are not organized by task for some reason - it looks like this:

#Generated by SJQv4 agent
#Mon Feb 20 19:11:52 EST 2012
task.mvfile.rcmax=0
task.mvfile.maxtimeratio=1.0
task.test1.maxtime=86400
task.rwcnametest.maxprocs=1
task.rwcnametest.testargs=mpg
task.sdcomskip.rcmax=1
task.outfile.resources=100
task.outfile.maxprocs=1
task.mvshowfolder.exe=script\:c\:/GroovyFiles/mv_show_Folder.groovy
task.sysmess.test=
task.dailyemail.testargs=
task.hdcomskip.schedule=ON
task.test1.resources=100
task.test1.exe=c\:/temp/encftp.bat
task.rwcnametest.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT"
task.xcode.exe=c\:/temp/dummy.bat
task.xcode.rcmax=0
task.mvfile.schedule=ON
task.renamese.resources=100
task.mvshowfolder.rcmax=0
task.filechecker.rcmax=0
task.prockids.args=
task.sysmess.maxprocs=1
task.mvshowfolder.maxtimeratio=1.0
task.sysmess.schedule=ON
task.hbipad.rcmin=0
task.outfile.testargs=mpg
task.hdcomskip.maxprocs=2
task.test1.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT","$SJQ4_LAST_SEGMENT"
task.sdcomskip.maxtimeratio=1.0
task.dailyemail.rcmin=0
task.mvfile.resources=100
task.sysmess.resources=100
task.hbipad.args=
task.sysmess.exe=script\:c\:/GroovyFiles/SysMess.groovy
task.rwcsd.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT","$SJQ4_LAST_SEGMENT"
task.outfile.maxtime=86400

I am also surprised that the file hasn't been update since Feb.

[Slugger]

Yeah, java sorts the props file in some unknown/weird way. It is only ever rewritten if there's a need to, such as when you define a new task for the client. If you don't make any config changes, it won't update it.

Well, if the props file looks right then it could be a bug with the web ui pages. Best confirmed by seeing if tasks are still being put into the queue and assigned out to the client as expected. You could also use the ui for the stv and see if the client config looks correct there.

[KarylFStein]

I use the the premiere alert script and that stopped working when going to Groovy 2 because the version of the script I had used a private class definition. It was a quick fix after searching this thread, but I didn't notice it was failing for several days.

Is there some way to get an alert in SageAlert if any job fails? I found how to generate alerts within a Groovy script earlier in this thread, but I don't see how that can apply in this "private" class case. I'm not a programmer, but I did find the try/catch construct and tried that. However, I can't include the class definition in that block, so that failure doesn't get "caught".

Anything else I can do--preferably a "higher-up" way to capture all failures in jobs and send some sort of notice to SA, (even if it just says check SJQ failed list).

[Slugger]

Originally Posted by KarylFStein

I use the the premiere alert script and that stopped working when going to Groovy 2 because the version of the script I had used a private class definition. It was a quick fix after searching this thread, but I didn't notice it was failing for several days.

Is there some way to get an alert in SageAlert if any job fails? I found how to generate alerts within a Groovy script earlier in this thread, but I don't see how that can apply in this "private" class case. I'm not a programmer, but I did find the try/catch construct and tried that. However, I can't include the class definition in that block, so that failure doesn't get "caught".

Anything else I can do--preferably a "higher-up" way to capture all failures in jobs and send some sort of notice to SA, (even if it just says check SJQ failed list).

https://code.google.com/p/sagetv-add...n_Task_Failure

But you'd have to add that option to every task manually. Depending on how many that is, this may suffice, and it's certainly the easiest solution.

Alternatively, you could write a groovy script that scans your job queue and generates an alert as newly failed jobs are discovered. This would be the "catch all" solution, but does require you to throw on your programmer's cap and write the scanner.

[KarylFStein]

Originally Posted by Slugger

https://code.google.com/p/sagetv-add...n_Task_Failure

But you'd have to add that option to every task manually. Depending on how many that is, this may suffice, and it's certainly the easiest solution.

Alternatively, you could write a groovy script that scans your job queue and generates an alert as newly failed jobs are discovered. This would be the "catch all" solution, but does require you to throw on your programmer's cap and write the scanner.

Ah, I missed that. Yes, it works great, thanks! Just one caveat...

If I use the web UI, (is the UI within SageTV no longer around?), to change one of a task's settings the sysmsgonfail setting disappears from the sjqagent.properties file and has to be manually re-added. I tested changing the required resources for one of my tasks.

P.S. If you set the required resources to 0 it shows up as 100 in the web UI. It says 0 in the sjqagent.properties file though.

[Slugger]

Originally Posted by KarylFStein

Ah, I missed that. Yes, it works great, thanks! Just one caveat...

If I use the web UI, (is the UI within SageTV no longer around?), to change one of a task's settings the sysmsgonfail setting disappears from the sjqagent.properties file and has to be manually re-added. I tested changing the required resources for one of my tasks.

P.S. If you set the required resources to 0 it shows up as 100 in the web UI. It says 0 in the sjqagent.properties file though.

Sage UI: Yes, but you have to install the SJQ STVi plugin within Sage to get it.*

Web UI: Bugs... 0 shows up as 100, most likely a bug. Losing the sysmsgonfail setting when editing through the web - bug/incomplete feature. The STVi UI will also do the same thing (i.e. overwrite the sysmsgonfail setting).*

* These UIs are incomplete. Tom is not actively developing/maintaining the STVi nor am I actively maintaining the web UI. To prevent loss of settings like this, you'll have to stick to manual configuration. Other bugs are also unlikely to be fixed (i.e. 0 showing as 100 on web). At this point in time, only catastrophic bugs have a chance of being fixed and I can't think of a UI bug that would ever fit into that category.

Edit: Or, for the web ui, provide me patches and I'll happily build them and put them in the repository. The web ui is written entirely in groovy. On the otherhand, if I ever hit the bugs you've hit and I need it fixed in my env then you may luck out and get the fix, but these days, my SJQ setup just runs in the background and I never touch it so I'm unlikely to hit these bugs (at least anytime soon).

[KarylFStein]

Well, I don't change things much these days. My problems lately have been with updates that I've installed blindly, (namely the license and groovy 2 things). Well, when I switched to CableCard Placeshifter chokes on maybe 50% of the time on playback, but guess that's a dead-end to try and solve *sigh*.

I have the SJQ UI installed, so not sure why it's not showing up in Placeshifter under settings but shows up locally on the HDxxx boxes. Editing the sjqagent.properties file manually is no big deal as I haven't changed the SJQ tasks in a long while.

I guess the lesson at this point is to just leave things alone .

[Skirge01]

Is there any issue installing the windows service on the same computer running SageTV? I had this with my old WHS v1 setup using SJQ3, but SJQ4 is giving me a "no route to host" message. If the client were installed on a different computer, I'd be checking firewall settings, but this is confusing.

[Slugger]

"No route to host" is some kind of network or firewall issue. There is no problem doing it, I do it myself. Just make sure you're not using 127.0.0.1, SJQ doesn't allow clients on the loopback interface. That's not the reason for that error, the error you're seeing is from the OS, it's not from SJQ.

If you're using a hostname, just try using the IP instead when configuring the client. Could be a DNS issue.

[Skirge01]

I started checking all the logs in the agent folder and found an issue referencing port 1098. As the log was named sagex-api.log, it's pretty clear it's related to that api. I set up a NAT rule for 1098 and that fixed the issue, but I'm trying to figure out what instructions/guide I was reading when I overlooked that step. I went back and couldn't find anything about forwarding port 1098 in the SJQ guide and I don't ever recall being instructed to read a guide on sagex.

Since others have gotten this running, I'm assuming that the issue was with me not following directions somewhere, so I'm really interested in knowing how I missed this.

[Slugger]

The sagex remote services uses port 1098. I assume sagex services docs would talk about that requirement. My plugins that use it assume that the remote services plugin is installed and working properly so none of my docs will explicitly talk about that plugin's installation, setup, or configuration.

[Skirge01]

Originally Posted by Slugger

The sagex remote services uses port 1098. I assume sagex services docs would talk about that requirement. My plugins that use it assume that the remote services plugin is installed and working properly so none of my docs will explicitly talk about that plugin's installation, setup, or configuration.

I figured as much, Slugger. Since nearly everything is installed through the plugin manager, reading instructions is pretty much obsolete at this point (except for yours, of course). I can't think what would have/should have pointed me to needing to open up that port, but I really didn't think it was SJQ. Oh well.

As it turns out, I'm still having issues with the Windows service agent. If I turn off my firewall on the Sage server (where the agent service is also installed), the agent shows green in Sage, but it apparently doesn't get the task info I entered in the properties file. I created only a comskip task and it doesn't show up for the agent. None of the logs currently show anything new. This is driving me nuts.

[Slugger]

Get the plugin version of the agent working on the same host as the SJQ server, it will at least help narrow down the problems. The task agent runs on its own port (23344 by default) and the server listens for traffic from agents on its own port (23347 by default). The agent port must be open on hosts running the agent and the server port must be open on the Sage server. At this stage, if it's a firewall issue preventing communication, I'd expect to see timeout errors logged on one side or the other, unless your firewall is not immediately dropping blocked packets, which may result in an infinite wait. That would explain why nothing's being logged.

[Skirge01]

I've got the plugin agent installed on the server. I opened up both of those ports (23344 & 23347) in pfSense to the Sage server and the Windows firewall is still turned off on it.

I'm now seeing these messages:

[ConnSendQueue@76ee4c] Error communicating with server:java.lang.NullPointerException
[ConnSendQueue@76ee4c] Cleaning up c/s connection
[ConnSendQueue@76ee4c] NetworkManager CommuinicationFailure: : /192.168.1.100:51173 type=1
[ConnSendQueue@76ee4c] MsgSend thread terminating for /192.168.1.100:51173

[rsagetv99]

Hi slugger, I noticed my shows have not been getting comskipped and after further investigation it seems that I have a corrupt database.

This started around the time that my drive ran out of space and I deleted a bunch of LOB cache files - although I thought I did it when the system was down.

I have some full backups of my SageTV directory, so I should be able to restore if that is an option. Can you give me a quick run-down of the restore procedure?

2012-11-15 21:18:00,306 INFO  [TaskQueue]: Scheduling queue processor for ~8 seconds from now!
2012-11-15 21:18:08,306 INFO  [TaskQueue]: Running queue processor now!
2012-11-15 21:18:08,306 INFO  [TaskQueue]: Enforcing licensing restrictions... license is valid!
2012-11-15 21:18:08,306 ERROR [DataStore]: SQL Error
org.h2.jdbc.JdbcSQLException: General error: "java.lang.RuntimeException: page[227] data leaf table:56 entries:10 parent:40 keys:[438, 440, 442, 444, 446, 448, 450, 452, 454, 456] offsets:[1959, 1781, 1692, 1514, 1425, 1247, 1158, 980, 891, 713] parent 40 expected 42" [50000-145]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
	at org.h2.message.DbException.get(DbException.java:156)
	at org.h2.message.DbException.convert(DbException.java:279)
	at org.h2.server.TcpServerThread.sendError(TcpServerThread.java:186)
	at org.h2.server.TcpServerThread.run(TcpServerThread.java:139)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: page[227] data leaf table:56 entries:10 parent:40 keys:[438, 440, 442, 444, 446, 448, 450, 452, 454, 456] offsets:[1959, 1781, 1692, 1514, 1425, 1247, 1158, 980, 891, 713] parent 40 expected 42
	at org.h2.message.DbException.throwInternalError(DbException.java:226)
	at org.h2.index.PageDataIndex.getPage(PageDataIndex.java:233)
	at org.h2.index.PageDataNode.find(PageDataNode.java:164)
	at org.h2.index.PageDataIndex.find(PageDataIndex.java:270)
	at org.h2.index.IndexCursor.find(IndexCursor.java:136)
	at org.h2.table.TableFilter.next(TableFilter.java:310)
	at org.h2.command.dml.Select.queryFlat(Select.java:498)
	at org.h2.command.dml.Select.queryWithoutCache(Select.java:597)
	at org.h2.command.dml.Query.query(Query.java:257)
	at org.h2.command.dml.Query.query(Query.java:227)
	at org.h2.command.CommandContainer.query(CommandContainer.java:78)
	at org.h2.command.Command.executeQuery(Command.java:132)
	at org.h2.server.TcpServerThread.process(TcpServerThread.java:278)
	at org.h2.server.TcpServerThread.run(TcpServerThread.java:137)
	... 1 more

	at org.h2.engine.SessionRemote.done(SessionRemote.java:543)
	at org.h2.command.CommandRemote.executeQuery(CommandRemote.java:152)
	at org.h2.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:77)
	at com.google.code.sagetvaddons.sjq.server.DataStore.getPendingTasks(DataStore.java:556)
	at com.google.code.sagetvaddons.sjq.server.TaskQueue$QueueProcessor.run(TaskQueue.java:98)
	at java.util.TimerThread.mainLoop(Unknown Source)
	at java.util.TimerThread.run(Unknown Source)

[Slugger]

Originally Posted by Skirge01

I've got the plugin agent installed on the server. I opened up both of those ports (23344 & 23347) in pfSense to the Sage server and the Windows firewall is still turned off on it.

I'm now seeing these messages:

[ConnSendQueue@76ee4c] Error communicating with server:java.lang.NullPointerException
[ConnSendQueue@76ee4c] Cleaning up c/s connection
[ConnSendQueue@76ee4c] NetworkManager CommuinicationFailure: : /192.168.1.100:51173 type=1
[ConnSendQueue@76ee4c] MsgSend thread terminating for /192.168.1.100:51173

Not sure what's producing those errors, but that's not coming from any code I've written. Starting to wonder if there's a port conflict on the system (i.e. something else running on the same port my plugins are trying to use)? Quick googling suggests that it's some other plugin producing it. Perhaps something using Apache ActiveMQ?