Plugin: SJQv4

[Slugger]

Originally Posted by Skirge01

With just one task running, it ran from 15:21 to 16:51 and 15 seconds after finishing, it started it again. I'll post back the times for this latest run to see if it's the exact same 90 minutes. If it holds true, that would mean it should be done stop at 18:21.

This sounds like you're using ShowAnalyzer or Comskip? One of them returns non-zero on success and if you don't tell SJQ that then it will treat it as failed. But if it returns 1 and it's the direct exe call then that's why it goes back to the queue and restarts all the time. I want to say it's SA that does this, but I can't remember.

I am curious about something, though. Is there a way to know if SJQ is successfully killing the task? I ask because I could swear that I once killed a task after only a few minutes and it still ran to completion over an hour later.

A java quirk... Java can only kill the direct exe it started, so if you're calling an exe from a groovy script, for example. Well, killing the task in SJQ is only going to kill the groovy process. Any processes that groovy started cannot be killed (from Java). So, yeah, if you're indirectly calling exes from other exes then the kill feature in SJQ isn't going to work. Nothing I can do about that.*

* Well, that's not true, I could write a native wrapper to launch exes, but then I would have had to write one for Windows and Linux (and OS X, which would have been hard because I have no OS X env). I didn't want to support all that native code so I didn't do it. Now that the project is EOL, it definitely won't be done. So, in short, nothing I can do about it (via Java).

[Skirge01]

Originally Posted by Slugger

This sounds like you're using ShowAnalyzer or Comskip? One of them returns non-zero on success and if you don't tell SJQ that then it will treat it as failed. But if it returns 1 and it's the direct exe call then that's why it goes back to the queue and restarts all the time. I want to say it's SA that does this, but I can't remember.



A java quirk... Java can only kill the direct exe it started, so if you're calling an exe from a groovy script, for example. Well, killing the task in SJQ is only going to kill the groovy process. Any processes that groovy started cannot be killed (from Java). So, yeah, if you're indirectly calling exes from other exes then the kill feature in SJQ isn't going to work. Nothing I can do about that.*

Correct. It's COMSKIP which does this and that's what I'm running. It returns a 1 if it succeeds, which means it found commercials and a 0 if it doesn't find any commercials.

Is there something I can change to get this to work properly? I see a "return 1" and 0 in the groovy script, but I thought those were related to groovy's success or failure in its test script.

EDIT: Here's the section I was referring to:

    
    def command = ['C:/comskip/comskip.exe', '--ini=C:/comskip/comskip-PRIMARYHD.ini', file].execute()
    command.consumeProcessOutput(System.out, System.err)
    command.waitFor() 
    if (command.exitValue())
        return 1 
    else
        return 0

* Well, that's not true, I could write a native wrapper to launch exes, but then I would have had to write one for Windows and Linux (and OS X, which would have been hard because I have no OS X env). I didn't want to support all that native code so I didn't do it. Now that the project is EOL, it definitely won't be done. So, in short, nothing I can do about it (via Java).

LOL! Understood. It's not a big deal anyhow, but I was curious what behavior I should be expecting.

[Slugger]

Capture the return code of the exe (via Process.waitFor(), for example). Check if it's 1 then return 0 from the groovy script, if it's 0 then return 0 as well (no commercials = ok), anything else is an error so just return that value and let SJQ sort it out.

[Skirge01]

Originally Posted by Slugger

Capture the return code of the exe (via Process.waitFor(), for example). Check if it's 1 then return 0 from the groovy script, if it's 0 then return 0 as well (no commercials = ok), anything else is an error so just return that value and let SJQ sort it out.

Is it as simple as this?

    def command = ['C:/comskip/comskip.exe', '--ini=C:/comskip/comskip-PRIMARYHD.ini', file].execute()
    command.consumeProcessOutput(System.out, System.err)
    command.waitFor()
    if (command.exitValue() <= 1)
        return 0
    else
        return command.exitValue()

Sorry for needing the hand holding, but I'm no programmer and have zero experience with Java. I sincerely appreciate all your help.

[Slugger]

Feel like you're missing a step in there...

def command = ['c:/comskip.exe' ...]
def p = command.execute()
p.consumeProcessOutput(System.out, System.err)
def rc = p.waitFor()
if(rc == 0 || rc == 1)
   return 0
else
   return 1

I know you have no experience and don't know Java, but I can't keep writing every bit of code (for everyone, not just you). Initially, SJQv4 attracted dozens of emails a week asking for "a little help", or "some sample code". I picked a few and did what I could, but then I had to cut it off. I chose those who showed an effort, such as you, so I'm happy to help, but I can't do it for ever. My plugin provides the toolbox, it's up to the user to utilize the tools provided. Initially, it was too many requests and not enough hours in the day, now it's more the fact that the toolbox I've created is at its end of life -- I don't support this stuff anymore because I, believe it or not, got out of the Sage dev game awhile ago. I'm actually working on an EPG plugin, but that's all I'm actively working on. The big problem I face now is that I barely remember how the internals of most of this work. Even though I wrote it, it's been so long since I've touched it, even my own setup, that when I get asked questions, I usually have to take 10-15 mins and go read the source code to figure out how things work.

With that said, I'll still gladly answer questions, but just be aware that the code snippets are going to be few and far between. Most of what you're looking for is available by googling it. It's a lot more work, but it's all out there. I will tend to still provide code examples for Sage specific things, but for general Java/Groovy questions, I will usually refer people to Google. Capturing the return code of a launched process, as you might imagine, falls into the general Groovy category.

Happy scripting!

[Skirge01]

Thanks again, Slugger. I searched for about 30 minutes before posting again because I (understandably) expected a post like that to be coming soon. I had also been hoping some others might jump in, so it wasn't just you trying to help me. It wasn't until I was utterly confused and frustrated that I gave up. Once I felt like an absolute idiot for not understanding anything I was reading, I came back. I nearly just said "screw it" and went about figuring out how to reinstall SJQ3. I still think that might be the best course of action. This groovy/java stuff is way over my head and I don't think I've got the background to figure it out.

I very much appreciate all the help you've given me already, but I'm not sure SJQ4 is the right path for someone who isn't a programmer. Not being able to do things the way I want is the reason I left Tivo and other STB's behind and went with Sage. At the moment, I feel pretty stupid, so it's clearly time to step away. Maybe after the holiday I'll do some further reading and see if I can manage to grasp any of it.

[Slugger]

If your use case for SJQ is very narrow (i.e. you just want to only ever run comskip) then v3 will suffice. But if you ever think you might want to do something else at some point then v4 is where you need to be. v3 is so limited in functionality compared to v4, but, as you're learning, it requires a little more understanding of scripting.

Trust me, if you were a "wizard" in v3 then you can use v4. It's not so much a lack of understanding, it's just that this v4 toolbox (i.e. Groovy) is so much bigger/powerful that you just need to step back and understand where to find the info you need. If you can write v3 rulesets then you can write groovy scripts to do the same thing. The difference is that in v3 I abstracted a lot of the details out of actions at the cost of limited functionality. Because I abstracted things so much, the only way newer things could be added to v3 is if I added them, which is what limited the functionality, ultimately.

Now in v4, I just piggyback on a full blown scripting language and say "go nuts". This means you can do almost anything you can imagine, but it also means you need to write the full script, start to finish. A tradeoff, but, imho, one for the better. Anyway, I guess my point is that I don't think you're too far off with Groovy and in the end you're going to benefit more from it (probably), but ultimately I guess you need to use the tool that you're most comfortable with.

Good luck and happy scripting!

[bikesquid]

Hey Slugger, got a license question....
I badly disfigured my sage.properties and had to revert to an earlier date... finding later other jacked up files,... disk started to fail on me. I've got everything sorted I think, but the license server thinks I'm unlicensed.... any suggestions on getting it to recognize the license I have? The message from the log is sjq4: The registered email ['xxx@yyy.com'] does not match the email in the license file!" I'm not sure where it's getting xxx@yyy.com or what it has on file... and neither is likely to be a current email address.

.... any suggestions on getting it sorted out?

B.

[Slugger]

All I have in my database is the email address you used at the time of purchase, the Paypal transaction id, and the plugin id(s) you purchased. If you don't know which email address you used, but can provide me (via PM or email) the Paypal transaction id then I can tell you which email address I have for you. If you know which email addresses it could be, try feeding them into the license retrieval web page. If you guess correctly, it'll email you a new license file, which you can ignore, you just need to figure out which email address is encoded in the license file you have and feed that same address to the license server config.

The other possibility is that the backup of your license file is old. New license files were issued a couple months back and only that version of the lic file will be accepted by the current version of the license server. In that case, the current email you're feeding might be correct, you just might need a new license file, which can be obtained at the link above.

[bikesquid]

pm sent w/email address used... though I have no access to that account, I think therein lies the problem....
thx.
B.

[wayner]

To pick up a discussion from post 1266 on Nov 14 (I have been out of town for a couple of weeks), I am having major problems with SJQ. As I said then I have the same task duplicated 8 times, all of my other tasks have disappeared - this is the case from both the UI within Sage and also the web UI. When I try to delete these duplicates once I press Commit Now the duplicate tasks that I just deleted come back and once again I have 8 of the same task.

There is other weird stuff to, as in when I select a TV file and click on Assign SJQ Tasks, I can select all of the tasks that used to exist but disappeared. But these tasks never run since no client is assigned.

I run WHS so I have backups. What files or folders should I restore? Should I restore the entire \SageTV\plugins\sjq and \SageTV\plugins\sjq folders? Or is there something else I could try?

This problem may have been caused by my system disk running out of space due to a lobs.db issue - perhaps that caused the SJQ files to be corrupted, but everything else seems fine on my system.

[bikesquid]

Originally Posted by wayner

To pick up a discussion from post 1266 on Nov 14 (I have been out of town for a couple of weeks), I am having major problems with SJQ. As I said then I have the same task duplicated 8 times, all of my other tasks have disappeared - this is the case from both the UI within Sage and also the web UI. When I try to delete these duplicates once I press Commit Now the duplicate tasks that I just deleted come back and once again I have 8 of the same task.

There is other weird stuff to, as in when I select a TV file and click on Assign SJQ Tasks, I can select all of the tasks that used to exist but disappeared. But these tasks never run since no client is assigned.

I run WHS so I have backups. What files or folders should I restore? Should I restore the entire \SageTV\plugins\sjq and \SageTV\plugins\sjq folders? Or is there something else I could try?

This problem may have been caused by my system disk running out of space due to a lobs.db issue - perhaps that caused the SJQ files to be corrupted, but everything else seems fine on my system.

Check your license file... in the sjq configuration screen it should tell you if you're licensed or unlicenced... this happend to me too, if your email hasn't changed since you bought your license follow the retrieval link a couple of posts back and you should be up and running... if that's your problem.

[wayner]

My license file was good- but I have had to roll back to an older hard drive from a couple of months ago. When I get the chance I will restore to a newer version so I may need a fix for this issue yet.

[Slugger]

As was done a few posts earlier, if you're seeing weird behaviour then the path to cause the least amount of frustration is to save the props file for each of your clients then toast the sjq db and force sjq to create a new, empty one. Tracking down a corrupted db probably isn't worth the hassle. The best bet of repairing such a situation would be full blown debug builds along with H2 db troubleshooting. Unfortunately, at this stage, debug builds of sjq with the debug logging necessary just aren't going to happen. You may want to investigate (via google, etc.) about how to diagnose, troubleshoot and repair an H2 database. Honestly, though, as I said, I'd salvage your task client props files and toast the db and start over.

[PLUCKYHD]

Slugger,

Should

println "my messages suck"

be printing out in default sage logs when passed through a groovy script in SJQ? Not seeing any of my logging so just checking with u before I dig deeper.

[Slugger]

Nope, those statements would appear in the SJQ logs for the task. So you'd need to install the SJQ STVi or the SJQ web UI plugins then view the task specific logs to see those statements. Nothing from SJQ groovy scripts would be logged to any Sage logs (i.e. sagetv_0.txt, etc.).

[bikesquid]

I've now built a new sage box after my puff/fizzle hard disk incident and have everything up and running on a fresh win7x64 install, and fresh sage install. Got all the plugins installed and have license showing as valid.

I do have access to a back up of all the sage files, and did copy the plugins directory structure over. I've got the server-client showing online but it's only showing one supported task rather than the 6-7 that I'd setup... any way to pull those other tasks over from the old setup?

EDIT: Never mind, patience my son.... It seems to have found 'em.

[PLUCKYHD]

Originally Posted by Slugger

Nope, those statements would appear in the SJQ logs for the task. So you'd need to install the SJQ STVi or the SJQ web UI plugins then view the task specific logs to see those statements. Nothing from SJQ groovy scripts would be logged to any Sage logs (i.e. sagetv_0.txt, etc.).

Thanks slugger wasn't sure if you were sending everything to SQJ or not helps allot thank you!!

[graywolf]

Originally Posted by Slugger

Let's take SJQ out of the equation (since SJQv3 cannot use Groovy directly)... I've been meaning to document this, but anyway here's a quick overview of my "groovy SageTV environment", which has nothing to do with SJQ.

1. Install Groovy 1.8; grab the Windows installer version, ensure Groovy is on the system path.

2. Grab the latest sagex-api.jar for your version of SageTV. Add it to the %HOME%/.groovy/lib folder. Doing this means groovy has access to the sagex APIs everytime it's started.

(NOTE: This works on Linux, too, I've done it on my test server, just do the equivalent "Linux" thing for the two steps above.)

3. Now simply write a script against the sagex-api.jar. You can now use all the Sage APIs from groovy scripts (SJQ not requred).

In my env I've written a bunch of utility scripts that I use often to set system properties, read props, etc. during plugin development. Then I wrote a wrapper around those such that now I have this program called "sageg" that allows me to run these scripts from my command line:

C:\Users\slugger>groovy --version
Groovy Version: 1.8.4 JVM: 1.6.0_20

C:\Users\slugger>sageg
sageg: Required command missing!  Valid commands are:

   getcp: Get a client property: <ui_context> <name>
   getos: Display the SageTV server's OS
   getsp: Get a server property: <name>
lnimport: Import a video and link it to a Show object
   setcp: Set a client property: <ui_context> <name> <val>
   setsp: Set a server property: <name> <val>

C:\Users\slugger>sageg getos
LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
Connecting to SageTV server at [192.168.0.42]
Windows XP

C:\Users\slugger>sageg getsp SageSync/ts
LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
Connecting to SageTV server at [192.168.0.42]
1326346500001

C:\Users\slugger>

This output is from my Windows 7 laptop, where I do a lot of Java coding, and it's talking to my Sage server on XP. With a single command line switch I can immediately start running the same commands against my Linux test server.

Each command listed above is just a plain old groovy script named after the command (i.e. getos.groovy) and are sitting in a predetermined directory. Some are just one liners (getos), others are a little longer (lnimport).

All those commands are various groovy scripts I've written to accomplish each task listed. As time goes on, I'll continue to add to that. You can do that same kind of thing once you install groovy and add the sagex-api jar to the classpath.

This will work with any version of Sage - just make sure the sagex-api jar is in the classpath and the RMI service is enabled with the sagex-services plugin on your Sage server.

NOTE: That if you then wanted to add groovy to SJQv3, you could do it by calling groovy.exe in your tasks. Remember that doing it this means you don't get any of the implicit variable injection as discussed in the SJQv4 docs (but all those auto variables are easily pulled into scripts with basic sagex API calls).

Struggling with this.

Would you be willing to share your sageg & lnimport code for me to use as examples?

Think with those 2 as examples, I think I would be able to figure out anything else I'd like to do with it.

[Slugger]

Originally Posted by graywolf

Struggling with this.

Would you be willing to share your sageg & lnimport code for me to use as examples?

Think with those 2 as examples, I think I would be able to figure out anything else I'd like to do with it.

I might be able to package something together, but it would only be with the understanding that it comes with absolutely zero support. I will warn you the code is not pretty and may not even be usable unless you know how it works. It is something I've hacked together over the last couple years and I never intended it for public release so the code quality reflects that. If you can live with that then I can probably package something up on the weekend.