Plugin: SJQv4

[awhenry]

Could anyone provide an example of how media information can be passed to the task executable if that information is not available via pre-defined environment variable? Specifically, I have created a groovy script that will be run via the cron function that will assign tasks to media objects based on various criteria. The task executables are Windows batch files. The first argument to the batch file is the path and file name of the media file, and that information is available via "$SJQ4_PATH/$SJQ4_LAST_SEGMENT". However the second argument needs to be the information contained in the MediaFileAPI.GetMediaFileFormatDescription string, which is not available as an environment variable. In SJQv3 I could use "$.GetMediaFileFormatDescription{}", but I do not think that will work in the new version.

Thanks.

[Slugger]

Originally Posted by awhenry

Could anyone provide an example of how media information can be passed to the task executable if that information is not available via pre-defined environment variable? Specifically, I have created a groovy script that will be run via the cron function that will assign tasks to media objects based on various criteria. The task executables are Windows batch files. The first argument to the batch file is the path and file name of the media file, and that information is available via "$SJQ4_PATH/$SJQ4_LAST_SEGMENT". However the second argument needs to be the information contained in the MediaFileAPI.GetMediaFileFormatDescription string, which is not available as an environment variable. In SJQv3 I could use "$.GetMediaFileFormatDescription{}", but I do not think that will work in the new version.

Thanks.

If you're running via cron then no env vars are ever going to be available. You'll have to find the media file of interest via groovy then pull all the info from the Sage API. So assuming you've found the media file of interest and stored it in a variable called mf:

String desc = MediaFileAPI.GetMediaFileFormatDescription(mf)

Then use the desc variable as an argument to the batch file you're calling (presumably from groovy). You'd pull the path and file names in similar fashion using the MediaFileAPI and pass those as arguments to the batch file as well.

[dadavo]

Originally Posted by Slugger

I'm starting to run out of ideas. For some reason, Sage doesn't appear to be loading the groovy jar file at start up. Enable Sage debug logging, if not already on. At the start of the sage log file (sagetv_0.txt) it will list all of the jar files it's loaded into its classpath. Does the groovy jar get listed? The problem is that the JRE is not loading the groovy jar and therefore doesn't recognize groovy as a supported scripting language. Get Sage/the JRE to load that jar file properly and the problem should be fixed. Why isn't the jar being loaded? That's the real mystery. The SageTV classpath from the sage debug log might start to lead us to an answer...

I have attached the log file. Thank you.

Neither upper or lowercase extension worked.

[awhenry]

Originally Posted by Slugger

If you're running via cron then no env vars are ever going to be available. You'll have to find the media file of interest via groovy then pull all the info from the Sage API. So assuming you've found the media file of interest and stored it in a variable called mf:

String desc = MediaFileAPI.GetMediaFileFormatDescription(mf)

Then use the desc variable as an argument to the batch file you're calling (presumably from groovy). You'd pull the path and file names in similar fashion using the MediaFileAPI and pass those as arguments to the batch file as well.

I apologize for being unclear. The cron process only queues tasks for media files. This is based on your find_tv_to_archive script, at least in terms of the queuing of tasks. My goal is to just queue up tasks rather than have the cron script do the work as I don't think there is a way to leverage resource allocation and multiple clients using the cron method alone since cron only runs on the server. Once queued, presumably clients capable of running those tasks would pick them off the queue and run them as resources are available. It is the client task definition that I am wondering about. The client task executable would be a Windows batch file in my case (called directly from the client task definition rather than wrapped in any groovy script), I just don't know how to collect and pass arguments to it that are not exposed through environment variables.

Hope this makes more sense this time.

Thanks.

[Slugger]

@awhenry Yeah, I see what you're saying. I need to support dynamic editing of task arguments, it's the only way you'll be able to do it. You'll probably want to ticket this issue to remind me. With that said, this is a big enough change that it's going to be some time (I'm still not quite out of vacation mode - as far as my Sage dev work goes). Until this feature is added, the only way to do it is as I've described by wrapping your batch file call within a groovy script and pulling the required args via groovy and calling your batch file.

[phelme]

Slugger, a couple of rambling questions on cron based tasks. Getting some weird behavior and need clarification on the way some things work these beasties.

I'm trying do a mediashrink task on watched shows that have been sitting around a while. I'm testing this through "Queue Task" and not actually using cron yet.

When I run the task without the actual p.waitFor(), process execution, the test script log tells me everything is going well and ends up in the Completed Task list. However, when I actually fire off mediashrink, it proceeds as expected, handbrake runs and finishes, the mkv is produced, but the test script log file is empty except for "The process was killed by SJQ." But here's the really odd thing, the task is executed again! With mediashrink recompressing the mpg (obviously I haven't deleted the MPG yet in the script). This will repeat ad nasueum until I manually kill off mediashrink (and after many attempts) remove the task from the queue. Why is it behaving so differently simply because i'm executing an outside command?? And why does it repeat itself? The only change is adding the cmdline, execute() and waitFor(). Eventually there will be tags on the metadata and all that, but it still shouldn't refire itself like it appears to be doing. I imagine it has something to do with the way the script is returning. I'm not sure it's even coming back to my script to be honest.

BTW, sometimes it can be a real pain to kill off a task and I have to disable the client to do it. );

Is it wise to do cron tasks without an executable stated for the task? Is that even allowed? Or would it be better to set an environment variable (though I think you said that can't be done within cron jobs) and pass it up to the executable that way?

And one more, in a cron task, what value needs to be returned and where to have a task marked "Completed" and not "Failed" or "Returned"? Haven't figured that out yet either.

[tmiranda]

Originally Posted by Spectrum

I've noticed that when going through the queued and completed tasks using the SJQ UI plugin that the entries will sometimes just disappear. It usually happens when scrolling up/down using the mouse wheel and it has persisted from initial install across several restarts of the SageTV service and client. It doesn't seem to affect task execution, but when it happens I have to back out of the current menu then re-enter it causing it to refresh. I made a viedeo screen cap of what is happening and through it up on a box at school. http://web.eecs.utk.edu/~rdeaver1/fi...s_Disapear.avi (~2.5MB)

I didn't see this listed as a current issue on the google code site for SJQ and couldn't find any issues re SJQ UI on tmiranda's google code site so I thought I would bring this up here to see if anyone else is seeing this or if slugger/tmiranda already know about it before logging it.

If needed I can provide logs, just let me know which logs you want

The video was very useful in helping to diagnose the problem, thank you. I see what is happening and I know why. I think I have a fix for it but will not be able to actually do it until the weekend.

[phelme]

made some progress. setup an empty batch script in as an executable, that solved the "Completed" task status.

but still having issues with running mediashrink from within the groovy script as a process.

It is not completing from within the script when given something to process. if I don't give it any parameters, it errors out and I get all the output, captured in consumeProcessesOutput(). But If pass in something to shrink, the compressing occurs, an MKV file is produced but my script is never returned to, almost like SJQ isn't waiting long enough (BTW, I have Max Time set to 1 day). Also it looks like Java no longer has control of mediashrink as it's out of the Java process tree when it starts to run Handbrake. And because the script hasn't "finished" , SJQ just calls my task all over again. And again and again.

I'm missing something somewhere. I'm beginning to think this isn't going to work unless I pass the media file up to MediaShrink as an executable as part of the task.

Anyone else trying to run mediashrink as a process in their script?

[Spectrum]

Originally Posted by bikesquid

I'd like to do a channel test and skip if it matches the recording.... I've tried several things but I obviously just don't get it.... Any help appreciated!

I have gotten that working and posted my comskip_test.groovy here. It's pretty similar to what Slugger already gave you though

[Spectrum]

Originally Posted by tmiranda

The video was very useful in helping to diagnose the problem, thank you. I see what is happening and I know why. I think I have a fix for it but will not be able to actually do it until the weekend.

Glad I could help sort something out! And no worries, it's a minor annoyance, not a huge problem. Appreciate all the hard work you and slugger are doing on this!

[Slugger]

Originally Posted by phelme

made some progress. setup an empty batch script in as an executable, that solved the "Completed" task status.

but still having issues with running mediashrink from within the groovy script as a process.

It is not completing from within the script when given something to process. if I don't give it any parameters, it errors out and I get all the output, captured in consumeProcessesOutput(). But If pass in something to shrink, the compressing occurs, an MKV file is produced but my script is never returned to, almost like SJQ isn't waiting long enough (BTW, I have Max Time set to 1 day). Also it looks like Java no longer has control of mediashrink as it's out of the Java process tree when it starts to run Handbrake. And because the script hasn't "finished" , SJQ just calls my task all over again. And again and again.

I'm missing something somewhere. I'm beginning to think this isn't going to work unless I pass the media file up to MediaShrink as an executable as part of the task.

Anyone else trying to run mediashrink as a process in their script?

Let's see the script you're using. My best guess is that, if I remember correctly, MediaShrink is itself a wrapper that calls other exes as part of its process so what we have is groovy > bat > mediashrink > exe and somewhere in that link it's being broken such that groovy/sjq can't keep track of the process. That's only a guess though. Best place to start is to look at the groovy script you're using.

And because the script hasn't "finished" , SJQ just calls my task all over again. And again and again.

I'm not sure what you mean by this? If SJQ kills your process (or says it did) it shouldn't be running again. Unless you mean you run the script again and it finds the same media file to process again because it never properly completed the first time? I could understand that, but that is avoidable by tagging the media file with metadata to say you've already processed it so as not to repeat processing of the same media file.

[ewelin]

Originally Posted by tmiranda

The video was very useful in helping to diagnose the problem, thank you. I see what is happening and I know why. I think I have a fix for it but will not be able to actually do it until the weekend.

I was just about to report this as well, but I had a hard time recreating it... It happens for me on the queue more than the completed tasks.

[ewelin]

Originally Posted by Slugger

You would have to write a script that went looking for these. Eventually I'll find the time to write such an example, but in the mean time if anyone else comfortable with groovy wants to help out it'd be appreciated. A quick skeleton of what needs to be done is shown below:
.......

If I dive in and attack this code would you be willing to look it over before I implement it? I'm comfortable with other languages, just never coded with groovy. Also is there a good syntax highlighter for groovy and maybe a compiler that you can recommend?

[awhenry]

Originally Posted by Slugger

@awhenry Yeah, I see what you're saying. I need to support dynamic editing of task arguments, it's the only way you'll be able to do it. You'll probably want to ticket this issue to remind me. With that said, this is a big enough change that it's going to be some time (I'm still not quite out of vacation mode - as far as my Sage dev work goes). Until this feature is added, the only way to do it is as I've described by wrapping your batch file call within a groovy script and pulling the required args via groovy and calling your batch file.

Don't implement this on my account. I decided to re-implement my Windows batch file as a groovy script, which is working well now and as a bonus allows me to do some things that are difficult or impossible using batch files.

Thanks for the help nevertheless.

[awhenry]

Originally Posted by ewelin

If I dive in and attack this code would you be willing to look it over before I implement it? I'm comfortable with other languages, just never coded with groovy. Also is there a good syntax highlighter for groovy and maybe a compiler that you can recommend?

I needed the same thing. I settled on Eclipse with the groovy plugin, as I am fairly familiar with Eclipse. This combo actually worked extremely well for me. By including all of the JARs in the Sage TV JARs folder in my project as well as defining a couple of imports in my script, I was able to test all aspects of my script before deploying, including all of the Sage API stuff. Very handy. Indispensable, actually.

For anyone interested, I had to include these import lines to get my scripts running in Eclipse (and left them in to no ill effect when running on the task clients):

import com.google.code.sagetvaddons.metadata.Factory;
import sagex.api.*;

Additionally, I had to copy the somf.jar file from the Sage JARs folder to the lib folder for my client service install and add that to the classpath in wrapper.conf (this step presumably wouldn't be necessary for anyone using just the plugin client).

I am guessing that you would need to run eclipse on the same machine the Sage server is running on in order for this to work (this is what I did, but I did not try any other setup). Additionally, you will want the 32-bit version of Eclipse regardless of your platform (I did try the 64-bit one, and it had problems with the H2 database).

Hope this helps, and good luck!

[Slugger]

Originally Posted by ewelin

If I dive in and attack this code would you be willing to look it over before I implement it? I'm comfortable with other languages, just never coded with groovy. Also is there a good syntax highlighter for groovy and maybe a compiler that you can recommend?

Yes. I can't promise a (super) quick turn around on the review, but I will look at it. If it's a relatively small script then I'd say 24-48 hrs would be fair. Larger, more complex scripts will require a larger block of my time to review and might have to wait until a weekend, etc.

Originally Posted by awhenry

I needed the same thing. I settled on Eclipse with the groovy plugin, as I am fairly familiar with Eclipse. This combo actually worked extremely well for me. By including all of the JARs in the Sage TV JARs folder in my project as well as defining a couple of imports in my script, I was able to test all aspects of my script before deploying, including all of the Sage API stuff. Very handy. Indispensable, actually.

For anyone interested, I had to include these import lines to get my scripts running in Eclipse (and left them in to no ill effect when running on the task clients):

import com.google.code.sagetvaddons.metadata.Factory;
import sagex.api.*;

Additionally, I had to copy the somf.jar file from the Sage JARs folder to the lib folder for my client service install and add that to the classpath in wrapper.conf (this step presumably wouldn't be necessary for anyone using just the plugin client).

I am guessing that you would need to run eclipse on the same machine the Sage server is running on in order for this to work (this is what I did, but I did not try any other setup). Additionally, you will want the 32-bit version of Eclipse regardless of your platform (I did try the 64-bit one, and it had problems with the H2 database).

Hope this helps, and good luck!

I have the same setup installed and it certainly works. If you go looking through this thread you'll also see me discuss the Groovy GUI editor option, which is what I actively use. Doesn't do syntax highlighting, but it allows for easy, quick running of scripts. You will have to do a lot of what's discussed above as far as adding extra jars, etc. to make scripts that work in SJQ work in these envs as well. Eventually, I'll package up my modified Groovy GUI editor and release it (unsupported) for those looking for a quick SJQ/groovy editor.

As far as having to run these setups on the same box as Sage, that's not true. Since SJQv4 utilizes the remote APIs, you can run it from anywhere. You may get some unexpected results if you have more than one Sage server on the LAN with the remote APIs installed, but if you just have one server on the LAN then you shouldn't have any problems running it from any host within the LAN.

[gabrielcab]

I'm very shamed and sorry to ask in public but theres lots and lots of pages of information and I dont have much additional time to read them without first knowing what exactly SJQv4 do in simple way?

[Slugger]

Originally Posted by gabrielcab

I'm very shamed and sorry to ask in public but theres lots and lots of pages of information and I dont have much additional time to read them without first knowing what exactly SJQv4 do in simple way?

In one sentence, SJQ allows you to automate tasks associated with your SageTV environment. SJQ (v4, especially) is a scripting platform and task queue management system that allows users to script and automate task execution across multiple hosts. Plenty of docs and discussion available to read on here for more details.

[gabrielcab]

Originally Posted by Slugger

In one sentence, SJQ allows you to automate tasks associated with your SageTV environment. SJQ (v4, especially) is a scripting platform and task queue management system that allows users to script and automate task execution across multiple hosts. Plenty of docs and discussion available to read on here for more details.

What for is primarily used? to schedule TV recording? what else can a home user could use it for an HTPC experimental?

[Slugger]

I think the most popular use would be to selectively schedule comskip processing of certain recordings while skipping others. People use it in all kinds of ways. I use it for comskip and I also use it to selectively archive certain recordings and move them from my Sage server to my NAS. Others use it to transcode their recordings. Others use it to schedule "safe" restarts of their Sage tv systems. Basically, if there's a task you want to do periodically with your system, the SJQ platform helps with it.