Plugin: SJQv4

[Slugger]

Originally Posted by tmiranda

I think Sluggers "You don't" comment means you don't know the priority of the tasks if multiple tasks are assigned to a Favorite. You can assign multiple tasks, but you can't control in what order they occur.

Exactly... see above response for further details.

[Slugger]

Originally Posted by m1abrams

Ok restarting SageTV fixed the pingc error. Now when I try to run my comskip task I got the error that java.exe did not exist in the SageTV/SageTV/jre/bin/ when it tried to run the test script. Well java.exe does not exist there, so just to see what would happen if I put it there, I copied java.exe from my java bin to that location.

After that I get the error, Remote SageAPI is not functional. Can't user server: rmi://192.168.10.20:1098

I ran netstat -an to see if the server was listening on 1098 and it is not even though the sagex-services is configed for port 1098

Is RMI enabled in the sagex-services plugin? It needs to be. By default it's turned on, but perhaps you disabled it at some point? Check the plugin config for sagex-services to confirm. If it says it's enabled then check firewall settings, etc. SJQ needs access to the sagex-services RMI server to operate properly.

[Slugger]

Originally Posted by wayner

How do you call a batch file from a groovy script? Can you execute a Windows/DOS command like copy or move from groovy or do you have to call a batch file to do that - what I mean is can you do something like exec("copy abc.mpg xyz.mpg") the way that you can in some scripting languages?

You have to call those kinds of commands via cmd.exe - you can't call it directly. I don't actually do it - I use the Apache Commons libs to do it instead - so the exact syntax isn't coming to me, but you have to use cmd.exe (since commands like copy, move, etc. are shell commands in Windows, not actual executable programs). I'd recommend using a Java lib to do this kind of stuff instead. Just install the commons-io plugin on your task client host (assuming you're using the plugin task client). Then you can add this kind of stuff to your groovy scripts:

// Note: This is a complete example script, but with all error checking suppressed

import org.apache.commons.io.FileUtils

File src = new File("foo.mpg")
File target = new File("bar.mpg")

FileUtils.copyFile(src, target)

The advantage to using Apache Commons (or any Java lib) to do this kind of stuff is that now your groovy script works on any platform.

Just as an example, to run an exe in a groovy script you do something like this:

String[] cmd = ["comskip.exe", "foo.mpg"]
Process p = cmd.execute()
p.waitFor()
int rc = p.exitCode()
return rc

But for file IO stuff I wouldn't use commands and processes, but rather I'd use something like Apache Commons IO API. But, as always, either way will work.

[Slugger]

Originally Posted by phelme

| jvm 1 | 2011/01/01 17:40:30 | log4j:ERROR Attempted to append to closed appender named [sjqAgentApp].
| jvm 1 | 2011/01/01 17:40:30 | log4j:ERROR Attempted to append to closed appender named [sjqAgentApp].
| jvm 1 | 2011/01/01 17:40:30 | log4j:ERROR Attempted to append to closed appender named [sjqAgentApp].

Ever figure this out? I'm getting 3 lines of this a second.

Yeah, it's fixed in svn and will be fixed for the next build. You can apply the fix now by following the directions in this post.

[hchucky]

Slugger, I believe I see an issue with your example comskip_test.groovy script.

Here is the scenario...
- comskip task is setup to execute on RecordingStarted
- your test script (comskip_test.groovy) is in place
- user selects (plays) a show from the program guide, then after a few seconds stops watching the program.

In the above scenario the comskip task will be executed however the test script will return 1 because mf=null. My guess is that it returns null because the media file is no longer present. This will put the comskip task back in the queue. Because the user is now longer watching the show (so no media) mf will always be null so this task will just sit in the queue in an endless loop.

Is there a particular reason you choose a return code of 1 vs 2 in your first check where you are validating variables/metadata? Seems like changing the return value to 2 would be a simple fix.

HC

[Slugger]

It's a tough call... I return the task to the queue because I assume if none of those vars exist then there was an error inserting the task into the queue. I further assume that if you put the task in the queue then you wanted to run it and if it doesn't then you'll want to retry running it until it finally is able to run (instead of just hiding it as a failed task).

With all of that said, I've modded the script to (better, but not perfectly) handle the case where the media file goes missing. You can download the updated version and replace the original. I've just committed the update to svn so it should be there now for download (use the same link as referenced in the tutorial).

[wayner]

Originally Posted by Slugger

Assigning more than one task is a valid option. You can run multiple tasks in response to an event if the tasks are mutually exclusive or at least they don't depend on each other.

My strategy here was going to be:

1) Run Comskip (if show was recorded from a channel with commercials)
2) Transcode and place output file to separate archive folder based on show name and change filename to episode name.
3) Move edl to archive folder and rename to be consistent with mpg file (now avi file) name.

To increase the degree of difficulty I may run the show through VRD to remove the commercials from the file prior to transcoding. That would negate the need to move an edl file.

[Slugger]

Originally Posted by wayner

My strategy here was going to be:

1) Run Comskip (if show was recorded from a channel with commercials)
2) Transcode and place output file to separate archive folder based on show name and change filename to episode name.
3) Move edl to archive folder and rename to be consistent with mpg file (now avi file) name.

To increase the degree of difficulty I may run the show through VRD to remove the commercials from the file prior to transcoding. That would negate the need to move an edl file.

I'd definitely make this one task because you obviously don't want to be comskipping the file and trying to move it at the same time. A single groovy script (or batch file, etc.) would handle this situation quite nicely.

[awhenry]

Until a week or so ago I had been using SageTV v6 and SJQv3 quite happily. However, I decided the time had come to upgrade to version 7 (which I like quite well personally), so now I am on SageTV v7 and SJQv4.

I am currently attempting to port my SJQv3 tasks to v4. I was able to get my ccextractor task running fairly easily, as that could naturally be attached to the "recording started" event. However I am having some difficulty determining how to port some of my other tasks, which in broad terms use handbrake (via a Windows batch file) to compress shows if I havn't watched them within a few days of airing. The old tasks did such things as check the media age, match against the show title, check if the airing was HD or not, and pass media file format info to the batch script so it could do different things based on that info. It seems this type of task cannot be attached to one of the available events in v4, so I would need to use the cron function to run it periodically. As far as scheduling, that is fine with me. However, it seems that using cron there is no media info available, as the cron event is not associated with any particular Sage object. Do I need to write basically a Java program to somehow enumerate all the Sage media objects and iterate through them in order to implement these tasks? Or is there some facility or technique I am overlooking?

Thanks.

[Slugger]

Originally Posted by awhenry

Until a week or so ago I had been using SageTV v6 and SJQv3 quite happily. However, I decided the time had come to upgrade to version 7 (which I like quite well personally), so now I am on SageTV v7 and SJQv4.

I am currently attempting to port my SJQv3 tasks to v4. I was able to get my ccextractor task running fairly easily, as that could naturally be attached to the "recording started" event. However I am having some difficulty determining how to port some of my other tasks, which in broad terms use handbrake (via a Windows batch file) to compress shows if I havn't watched them within a few days of airing. The old tasks did such things as check the media age, match against the show title, check if the airing was HD or not, and pass media file format info to the batch script so it could do different things based on that info. It seems this type of task cannot be attached to one of the available events in v4, so I would need to use the cron function to run it periodically. As far as scheduling, that is fine with me. However, it seems that using cron there is no media info available, as the cron event is not associated with any particular Sage object. Do I need to write basically a Java program to somehow enumerate all the Sage media objects and iterate through them in order to implement these tasks? Or is there some facility or technique I am overlooking?

Thanks.

You don't need to write Java code, just Groovy script (though it looks a lot like Java).

for(Object mediaFile : MediaFileAPI.GetMediaFiles()) {
   /*
      If the media file matches the given title regex, is an HD recording
     and it's older than 3 days then enter the if block and do your thing
   */
   if(MediaFileAPI.GetMediaTitle(mediaFile) =~ /My Title|Another Title|Yet Another Title/ && AiringAPI.IsAiringHDTV(mediaFile) && System.currentTimeMillis() - MediaFileAPI.GetFileEndTime(mediaFile) > (3L * 86400000L)) {
      // Do whatever it is you want to do for the matching recordings in here

      // You should tag recordings you process with a metadata value that you also check for so as not to process the same recording more than once, if applicable
   }
}

[Omniver]

Originally Posted by Slugger

I'd definitely make this one task because you obviously don't want to be comskipping the file and trying to move it at the same time. A single groovy script (or batch file, etc.) would handle this situation quite nicely.

This is exactly what I'm looking for as well. Wayner, if you take this route please post an example of your script. I'll do the same if I get to it first.

[awhenry]

Originally Posted by Slugger

You don't need to write Java code, just Groovy script (though it looks a lot like Java).

for(Object mediaFile : MediaFileAPI.GetMediaFiles()) {
   /*
      If the media file matches the given title regex, is an HD recording
     and it's older than 3 days then enter the if block and do your thing
   */
   if(MediaFileAPI.GetMediaTitle(mediaFile) =~ /My Title|Another Title|Yet Another Title/ && AiringAPI.IsAiringHDTV(mediaFile) && System.currentTimeMillis() - MediaFileAPI.GetFileEndTime(mediaFile) > (3L * 86400000L)) {
      // Do whatever it is you want to do for the matching recordings in here

      // You should tag recordings you process with a metadata value that you also check for so as not to process the same recording more than once, if applicable
   }
}

Thanks for the pointer. I wonder what your opinion is on instead creating a task (or actually several tasks) that would attach to the "recording stopped" event and would (assuming other media tests succeeded) re-queue repeatedly until the media file was appropriately old, then run the handbrake batch file? On the plus side, this would avoid me having to continually rescan every media file when I am really only looking for a few (relatively) new arrivals to process. On the downside, I do not know what consequence there might be for perhaps multiple tasks constantly re-queueing over the course of many days, or if the tasks would properly handle the case where a file was deleted between runs. How much time elapses between a task requesting a re-queue and it running again?

As an extension to your example above, can you tell me what the syntax would be to execute a batch script, passing it the full media path and the "media file format description" (formerly contained in $.GetMediaFileFormatDescription{}) as arguments?

Thanks.

[Slugger]

Originally Posted by awhenry

Thanks for the pointer. I wonder what your opinion is on instead creating a task (or actually several tasks) that would attach to the "recording stopped" event and would (assuming other media tests succeeded) re-queue repeatedly until the media file was appropriately old, then run the handbrake batch file? On the plus side, this would avoid me having to continually rescan every media file when I am really only looking for a few (relatively) new arrivals to process. On the downside, I do not know what consequence there might be for perhaps multiple tasks constantly re-queueing over the course of many days, or if the tasks would properly handle the case where a file was deleted between runs. How much time elapses between a task requesting a re-queue and it running again?

As an extension to your example above, can you tell me what the syntax would be to execute a batch script, passing it the full media path and the "media file format description" (formerly contained in $.GetMediaFileFormatDescription{}) as arguments?

If you're looking to only process recordings once they hit a certain age then I'd definitely do it with a cron task. You don't want to be doing it on RecordingStopped. You'll definitely want to periodically scan your media and find those recordings that are old enough to be processed. As I said, you need to tag recordings that you've processed (maybe, depends if you're moving them or not). Either way you need to make sure that you don't reprocess the same file more than once. Note: All of this is untested and might need a little tweaking, etc...

for(Object mediaFile : MediaFileAPI.GetMediaFiles()) {
   /*
      If the media file matches the given title regex, is an HD recording
     and it's older than 3 days then enter the if block and do your thing
   */
   if(MediaFileAPI.GetMediaTitle(mediaFile) =~ /My Title|Another Title|Yet Another Title/ && AiringAPI.IsAiringHDTV(mediaFile) && System.currentTimeMillis() - MediaFileAPI.GetFileEndTime(mediaFile) > (3L * 86400000L)) {
      def cmdLine = ["C:/handbrake.exe", MediaFileAPI.GetFileForSegment(mediaFile, 0).getAbsolutePath(), MediaFileAPI.GetMediaFileFormatDescription(mediaFile)]
      Process p = cmdLine.execute()
      p.waitFor()
   }
}

Note: This example actually runs handbrake on each file as it is found. You might instead want to queue up a separate handbrake job for each file you find and then manage those through the queue separately. This script also does not tag files that have been processed so if you ran the script immediately after its first run then it'd end up processing the same files again. You will want to attach metadata to each file you process and then check that the tag doesn't exist before proceeding to run handbrake, etc. This example also does not do any error checking.

[awhenry]

Earlier in this thread somewhere I think I remember reading about you maybe releasing something that could be used to test scripts before deploying them, some kind of development environment where we could get some immediate feedback and do some debugging on scripts we are writing. Is there any news on that? The only alternative I am aware of is to take a crack at writing it, attach it to a task, hope for the best, and check the logs after the task runs. Cumbersome and slow, especially for newcomers to this scripting language.

Thanks!

[Slugger]

Originally Posted by awhenry

Earlier in this thread somewhere I think I remember reading about you maybe releasing something that could be used to test scripts before deploying them, some kind of development environment where we could get some immediate feedback and do some debugging on scripts we are writing. Is there any news on that? The only alternative I am aware of is to take a crack at writing it, attach it to a task, hope for the best, and check the logs after the task runs. Cumbersome and slow, especially for newcomers to this scripting language.

Thanks!

This post talks about how to install and invoke the Groovy IDE. The only issue is that all the SJQ and Sage API globals won't be available to your scripts, but it at least allows you to write generic groovy scripts and test them, etc. The Groovy IDE + SJQ/Sage env version will be coming eventually. I'm fighting a cold and just getting back to work after an extended vacation so it'll be a few more days before I find some quality Sage dev time.

[Spectrum]

I've spent the better part of 2 days converting from SJQ3-4 and while it wasn't exactly a smooth conversion, I do like the new model better. Good job, even though there is a learning curve!

One of the things I noticed when modifying the comskip_test.groovy script is that

SJQ4_METADATA.get("SJQ4_CHAN_NAME")

returns a null. Am I doing something wrong here, is that metadata property just not being set in this context, or is it something that is not enabled yet? I found a workaround in an earlier post using the AiringAPI, but it would be easier to use the metadata

I've attached my comskip_test.groovy that has been modified to allow skipping comskip based on channel name incase anyone is interested. Just edit the SkipList string to add matches for channels you don't want to perform comskip on.

[Slugger]

You'll have to pull the channel name via the AiringAPI. Only MediaFile metadata is attached in the context of RecordingStopped/Started events. Anything else will have to be pulled via the API, as you've done.

[Spectrum]

Ahh that makes sense. Now to let things run for a day or so and see if anything blows up

[broconne]

Finally making the migration from SJQ3 to SJQ4 and running into an issue - but not sure what the cause of the problem is.
I am attempting to setup comskip

I don't get anything in my sjqagent.log, in my wrapper.log I have quite a few entries complaining about Error: Attempted to append to closed appender.

If I look at the log from the test script from the SageUI I see that there is a ScriptException whose cause seems to be NumberFormatException: null - presumably meaning we attempted to parse a String, but null was sent in before. I copied the provided comskip test groovy file and I see the first action is to do a parseInt on SJQ4_ID - so I am guessing that is where I am failing. I can only assume that I am calling the script incorrectly?

Her eis what I have in my sjqagent.properties:

task.comskip.exe=C:/comskip/comskip.exe
task.comskip.args="$SJQ4_PATH\\$SJQ4_LAST_SEGMENT"
task.comskip.test=C:/sjqc/scripts/comskip_test.groovy
# task.mytask.testargs=
# task.mytask.schedule=* 13-17 * * *
# task.mytask.resources=50
task.comskip.maxprocs=1
# task.mytask.rcmin=0
# task.mytask.rcmax=0

Any tips on what I am missing?

[Slugger]

How are you queuing tasks? Recordings must be queued either via an event (RecordingStarted/RecordingStopped) or you must queue a recording manually from its recording details screen. That's how SJQ is able to set all those variables is uses in the script (SJQ4_ID, etc). If you just manually queue a comskip task from the SJQ menu then it won't know which recording it's for and therefore all of that metadata will not be properly set.