Plugin: SJQv4

[Slugger]

Originally Posted by barney B.A.

I have lots of plugins. I suppose it could be one of them, however I only ever had issues since installing sjq4. Could it be that there is another plugin not playing nicely with sjq4? I am willing to help any way I can. Be warned though that I am not a programmer and the extent of my scripting abilities ends with the dos prompt and notepad.

A bad interaction between plugins is possible - anything is possible with so many separate parts working together (i.e. many plugins). I mean the only way to track it down is to do a fresh install of just the core, run it for a few days and confirm that there is no signs of heap problems. Next, install SJQ by itself, use it, and see what happens. If there are no mem issues then start adding plugins one by one until we find the bad interaction, if one exists. Also, you'd have to install one plugin along side SJQ, use it for a few days then remove it and install the next, etc. There are just too many combinations once you start talking about more than a few additional plugins.

Part of the problem is that SJQ installs a lot of stuff as dependencies... Jetty and some jetty apps (remote APIs). It's a big beast on its own. Like I said, I stripped my system down to core + SJQ (and deps) and could not reproduce the issue. I've since added back some other plugins (SRE, SageAlert, various UI mods) and haven't had any issues since.

[matterw]

Just installed SJQ 4, set it up and getting the following error when a task runs "Exe does not exist or cannot be executed! [C:\Program Files\SageTV\SageTV\jre\bin\java.exe] in the Log for the Test Script...

Any suggestions?

Thanks,
Matt

[Slugger]

Originally Posted by matterw

Just installed SJQ 4, set it up and getting the following error when a task runs "Exe does not exist or cannot be executed! [C:\Program Files\SageTV\SageTV\jre\bin\java.exe] in the Log for the Test Script...

Any suggestions?

Thanks,
Matt

It would appear that the bundled JRE with SageTV is not laid out in a standard way. Where is the java.exe located under C:\Program Files\SageTV\SageTV\jre\? Either way, the expected path is hardcoded so I'm going to have to make a code change, but I'll need to know where it is.

[matterw]

Originally Posted by Slugger

It would appear that the bundled JRE with SageTV is not laid out in a standard way. Where is the java.exe located under C:\Program Files\SageTV\SageTV\jre\? Either way, the expected path is hardcoded so I'm going to have to make a code change, but I'll need to know where it is.

It's interesting. I checked the directory C:\Program Files\SageTV\SageTV\jre\bin and could not find java.exe (my guess it was using the java.exe in c:\windows\system32 since that was in the path), so I renamed the jre directory under SageTV and set JAVA_HOME to c:\program files\Java\jre6 and now I get a different message -- "LOG4J: Configured Loggin for: sagex-api using file : sagex-api.log4j.properties javax.script.ScriptException: java.lang.NumberFormatException: null"

Any suggestions on this one?

[Slugger]

Did you delete the recording that was queued? If not, how did you queue the task? View the metadata for that task. Is there an SJQ4_ID variable set? Actually, I know there isn't, and that's the problem. This either happens because the recording has since been deleted or the task was not queued in response to a recording started event nor from the recording details screen. If there is no metadata for the task, just delete it because it'll never work. Requeue it from the recording details screen if it hasn't been deleted. Ensure there is proper metadata assigned to the task (SJQ4_TYPE=MediaFile and SJQ4_ID=nnnnnn, where nnnnnn is a valid integer).

[matterw]

Originally Posted by Slugger

Did you delete the recording that was queued? If not, how did you queue the task? View the metadata for that task. Is there an SJQ4_ID variable set? Actually, I know there isn't, and that's the problem. This either happens because the recording has since been deleted or the task was not queued in response to a recording started event nor from the recording details screen. If there is no metadata for the task, just delete it because it'll never work. Requeue it from the recording details screen if it hasn't been deleted. Ensure there is proper metadata assigned to the task (SJQ4_TYPE=MediaFile and SJQ4_ID=nnnnnn, where nnnnnn is a valid integer).

That looks like it fixed it -- I just Queued the Task manually. After looking at the threads it seems that I have the same confusion as bikesquid:

(

Originally Posted by bikesquid

My client now seems to be doing a comskip on a NEW recording, probably explains the 'null' error I was getting. In the good ol' days of v3 it simply scanned the directory and tackled anything without the .edl... I liked that, how can I get that process concept back? AND how can I get it to do the things left unprocessed by post update black hole?

)

I am used to SJQ just scanning the whole directory, but now it seems that the paradigm has changed, so I need to get used to it.

One other piece -- how does SJQ take care of removing old *.edl files when the recording is removed. Is that another script?

******
Update:
******
Based on your response earlier in this thread, I found the answer to my question on how to clean up the old files...:

Originally Posted by Slugger

Best way to get the file cleaner: install my Sage Artifact Delete plugin. If you would rather implement it in SJQ then open a ticket and I'll write a tutorial on one way to do it (but it probably won't be before Christmas).

[matterw]

When I looked at the sjqagent.log file today I see quite a few "IOError
java.io.UTFDataFormatException" errors.

Attached is a piece of the SJQ Agent log file. Can you provide any reason or root cause for this error?

Thanks,
Matt

[Slugger]

Define "quite a few"? Always? Only when comskip is the task that's being run? Any other tasks? All tasks? If it's just comskip that's causing it then I'd say that output from comskip is causing it. If it's every task then I'd ask what version of Java you're running because the code that's failing is JRE code (I pass the output of the exe to a JRE class, which is then throwing the exception). If it's more than one exe, but not all exes then that's a whole different scenario requiring investigation.

Is there a matching error in the sjq.log file for the SJQ server at the same time as the client error? I wouldn't expect one, but I'd be curious to know if there is one.

With that said, tasks are still completing successfully, right? The logging of output is done separately from the actual command execution and so even though the logging is failing, the task should still be running properly and being marked as completed. Assuming that's true, then it's not a fatal error (other than task output is not being logged back to the SJQ db), but I'd still like some more details so that I can try to track it down if it's indeed a problem that I can address in my code.

[matterw]

Originally Posted by Slugger

Define "quite a few"? Always? Only when comskip is the task that's being run? Any other tasks? All tasks? If it's just comskip that's causing it then I'd say that output from comskip is causing it. If it's every task then I'd ask what version of Java you're running because the code that's failing is JRE code (I pass the output of the exe to a JRE class, which is then throwing the exception). If it's more than one exe, but not all exes then that's a whole different scenario requiring investigation.

Is there a matching error in the sjq.log file for the SJQ server at the same time as the client error? I wouldn't expect one, but I'd be curious to know if there is one.

With that said, tasks are still completing successfully, right? The logging of output is done separately from the actual command execution and so even though the logging is failing, the task should still be running properly and being marked as completed. Assuming that's true, then it's not a fatal error (other than task output is not being logged back to the SJQ db), but I'd still like some more details so that I can try to track it down if it's indeed a problem that I can address in my code.

It's happened 13 times since I left it last night... Seems like a lot to me. I attached both logs in an attempt to give you more insight into the issue. If there is more logging that could help, then let me know and I'll be more than happy to provide...

The JRE version is 1.6.0_23-b05.

/Matt

[Slugger]

Some research has yielded the likely cause of the issue. It seems UTF-8 encoded strings used in Java IO streams can only be 64K max in length. I'm assuming, from some limited testing/experience, the output of comskip is greater than 64K and is causing the issue.

You can star this ticket if you want to be informed of progress on the fix. I'm still assuming that tasks are properly completing and therefore this isn't a critical bug and so it will be fixed some time after the new year.

[tmiranda]

Originally Posted by Slugger

I'm assuming, from some limited testing/experience, the output of comskip is greater than 64K and is causing the issue.

@matterw - It will only generate that much if verbose is set to 10. Edit the comskip.ini and try setting verbose to 0 to suppress all of the text.

[toricred]

Is there a way to have the test script set a variable that is then passed to the executable? I'm trying to have the test script check the channel number and use different profiles in ShowAnalyzer based on the channel number. My theory is to set a variable that gets added as a command-line argument to the executable.

[Slugger]

There's no way to that now. As an alternative, you could write a property to wiz.bin in the test script (via Configuration.SetServerProperty()) and then read that prop back in the exe (only if the exe is itself a groovy script, you could wrap other exes in a groovy script, if necessary). Open a ticket to allow mods to exe cmd args via test script. Not sure how feasible it is, but if there's a ticket I'll at least investigate it (after the holidays).

[hchucky]

Slugger, I've been running v3 for a while. Recently I upgraded SJQ to v4 however v3 kept running. Today I decided to look into v4 further. I installed the v4 plugin client and GUI plugin. I uninstalled the v3 agent which was running as a Windows service.

I can still get to v3 through the web GUI. Shouldn't v3 have been cleaned up\removed (accept the v3 win agent) when v4 was installed?

Thx,
HC

[Slugger]

Originally Posted by hchucky

Slugger, I've been running v3 for a while. Recently I upgraded SJQ to v4 however v3 kept running. Today I decided to look into v4 further. I installed the v4 plugin client and GUI plugin. I uninstalled the v3 agent which was running as a Windows service.

I can still get to v3 through the web GUI. Shouldn't v3 have been cleaned up\removed (accept the v3 win agent) when v4 was installed?

Thx,
HC

Yes, if you installed v3 as a Sage v7 plugin. If you just carried over the v6 version of the plugin when upgrading to v7 then it would not have auto cleaned itself. If that's the case, just remove the sjq.context.xml file from SageTV/jetty/contexts/, that will uninstall and stop v3 from running.

[wayner]

Sorry, this has to be a dumb question but how do you edit tasks if you have made a typo and how do you delete tasks?

Also - by default comskip seems to return a code that indicates failure as far as SJQ is concerned. I fixed this in the past (ie. SJQ2-3) using a batch file that changes the return code - but now I am getting a "blue" bullet in completed tasks, even though it seems to have completed ok.

[hchucky]

Originally Posted by Slugger

Yes, if you installed v3 as a Sage v7 plugin. If you just carried over the v6 version of the plugin when upgrading to v7 then it would not have auto cleaned itself. If that's the case, just remove the sjq.context.xml file from SageTV/jetty/contexts/, that will uninstall and stop v3 from running.

Thanks Slugger!

HC

[hchucky]

Originally Posted by wayner

Sorry, this has to be a dumb question but how do you edit tasks if you have made a typo and how do you delete tasks?

Also - by default comskip seems to return a code that indicates failure as far as SJQ is concerned. I fixed this in the past (ie. SJQ2-3) using a batch file that changes the return code - but now I am getting a "blue" bullet in completed tasks, even though it seems to have completed ok.

You can edit tasks in the sjqagent.properties file located in the SageTV\plugins\sjq-agent\conf directory.

I think blue= good, but I may be wrong...this is all new to me as well.

HC

[Slugger]

Originally Posted by wayner

Sorry, this has to be a dumb question but how do you edit tasks if you have made a typo and how do you delete tasks?

Also - by default comskip seems to return a code that indicates failure as far as SJQ is concerned. I fixed this in the past (ie. SJQ2-3) using a batch file that changes the return code - but now I am getting a "blue" bullet in completed tasks, even though it seems to have completed ok.

Originally Posted by hchucky

You can edit tasks in the sjqagent.properties file located in the SageTV\plugins\sjq-agent\conf directory.

I think blue= good, but I may be wrong...this is all new to me as well.

HC

Blue is good. But you can also tell SJQ what return code range is acceptable via the task editor. Load up your clients in the GUI, then select the supported tasks and then edit them. There is a min/max return code setting, which is where you tell SJQ what the expected return codes are for the task. You can also edit the rest of the task's settings. You can also delete tasks from there as well.

Also, you can directly edit the sjqagent.properties file as well. You can edit this file while SJQ is running, there is no need to stop SJQ nor Sage to edit the file.

[hchucky]

Is there a way to disable the prompting for a Task and Trigger Event when setting up a manual recording?

HC