Plugin: SJQv4

[Slugger]

Originally Posted by SteveW

So if this is running as a cron job, I take it we don't have the ability as we did in v3 to run an immediate scan at the click of a button? I ask because sometimes I use that having had the task client off temporarily to reduce the load on my server, and then decide to fire it up and forced it to run right away. There have been a few other scenarios for me where that has been a handy feature...

You can force a scan on demand...

Setup > SJQ > Queue Task >

Then select the task name you created for the scanner script and then it'll scan right away.

[SteveW]

Hmm. I think I'm not defining the script path properly...

[Slugger]

You are correct:

2011-02-26 00:56:08,022 ERROR [192_166_1_100-23347-2]: Unable to read script '\\Steve-PC\Sage\Scripts\media_file_scanner.groovy'; marking task as FAILED!

Which ever client is trying to run the script can't find it at the given path. If the client is remote and you're using the Windows service standalone version of the task client, you must change the user the service runs as from the default LocalSystem account because the LocalSystem account cannot read files from the network. If that's not it then check the path name and ensure it's correct (by trying to load that script from Windows Explorer on the tast client host using the path as specified above.

The same problem would exist for the Sage plugin version if you're running Sage as a service using the LocalSystem account. LocalSystem account cannot access network resources.

[SteveW]

So if I wanted to have only one set of groovy scripts that all my clients use, rather than have a set on every PC, is it feasible to have those scripts at a share on the server? If my share where the scripts are is say called "gscripts", and my clients have the necessary accounts with network access:

\\servername\gscripts\scriptname.groovy

Is this supported as a location that all clients - sage and windows service alike can use?

[Slugger]

Definitely, I do it myself. All my groovy scripts are on a network share that all my task clients use. I use the Windows service, Sage plugin, and console task clients during testing. Sage plugin and console versions are run on both Windows and Linux. Windows uses network shares to access the script and Linux access via samba mounts - it all works for me. It's just a matter of properly configuring your file sharing on the network and making sure the task clients are running with sufficient permissions to access the scripts off the network shares.

[SteveW]

So that's all testing out fine now. I see results for "SCANFORMEDIA" task listed with a blue icon and indicating which shows passed the test, so all good there.

On your tutorial, your SDCOMSKIP task has the exe for the commercial detection (SAV for you, COMSKIP for me), an exe arg - "$SJ4_PATH....", test script, and test script arg. So based on that, more questions from me:

1) Since I'm using media_file_scanner.groovy script, I assume in my COMSKIP task that I would completely drop the test script and script test args since I've already run a script beforehand?

2) If my recordings directory is a share, is defined is Sage as a UNC path to that share, and if I have task clients outside of sage, then I do not have to wrap in a script for MAPDIR, I would use $SJQ4_PATH as you have in the example, correct?

3) Using COMSKIP.EXE, I a little confused on my min/max return code. Since COMSKIP returns a 1 on success, am I setting my min at 1? If so what would the max be set at?? 2?

Tnx

[Slugger]

Originally Posted by SteveW

So that's all testing out fine now. I see results for "SCANFORMEDIA" task listed with a blue icon and indicating which shows passed the test, so all good there.

On your tutorial, your SDCOMSKIP task has the exe for the commercial detection (SAV for you, COMSKIP for me), an exe arg - "$SJ4_PATH....", test script, and test script arg. So based on that, more questions from me:

1) Since I'm using media_file_scanner.groovy script, I assume in my COMSKIP task that I would completely drop the test script and script test args since I've already run a script beforehand?

If you look at the test script, it's testing for the "safety" of running comksip. It checks that the media file isn't actively recording, an edl doesn't already exist, etc. If you don't have any preconditions before starting comskip on a file then, yes, you can remove the test script and don't have to use it. Alternatively, if you have different preconditions for running comskip, then you can tweak the test script for your needs.

2) If my recordings directory is a share, is defined is Sage as a UNC path to that share, and if I have task clients outside of sage, then I do not have to wrap in a script for MAPDIR, I would use $SJQ4_PATH as you have in the example, correct?

Yes, if Sage uses UNC paths for all recordings and those UNC paths are valid on all your task client hosts then, yes, you can just use $SJQ4_PATH. Note, however, that the current task client version supports an agent.mapdir setting that allows you to specify the mapdir settings, if necessary, directly in the task client. You'd still have to use Groovy scripts to use that setting, but it's way better than a wrapper around exes to achieve the same effect. I only mention this for everyone else reading this. Based on your description of your setup, you should be fine just using $SJQ4_PATH.

3) Using COMSKIP.EXE, I a little confused on my min/max return code. Since COMSKIP returns a 1 on success, am I setting my min at 1? If so what would the max be set at?? 2?

Tnx

min = 0
max = 1

Why? Because comskip will return 0 if it processed a file successfully, but found no commercials and returns 1 if it processed the file successfully and found commercials. If you don't think code 0 is a successful code (i.e. you always expect comskip to find commercials) then use min = 1 and max = 1.

[Slugger]

Just a follow up on last week's discussion about upgrading sagex-services. 7.0.23.9 is good and users can upgrade to that version without breaking SJQv4. Thanks to stuckless for tackling the previous issues quickly, once reported.

[Slugger]

If you upgrade your task client(s) to 4.0.0.1403 then you MUST upgrade the server to 4.0.0.1404. The new task client is not backwards compatible with older servers. The opposite is also true (new server requires new client), but that's enforced automatically by the server.

Trying to use task client 1403 with a server older than 1404 will cause you all kinds of headaches. You've been warned.

Happy scripting!

[tmiranda]

I just uploaded a new UI Mod that adds support for mapdir.

[tmiranda]

@davephan - What happened to the post that contained the screenshots of how SJQ3 displayed task status? I was just going to start looking at that but the post is gone?

[wayner]

My .lobs.db folder takes up over 10GB of space on my Sage server - I am guessing that this is a problem. How do I fix this?

[davephan]

Originally Posted by tmiranda

@davephan - What happened to the post that contained the screenshots of how SJQ3 displayed task status? I was just going to start looking at that but the post is gone?

I found it. The post is over here:

http://forums.sagetv.com/forums/show...=45174&page=61

it is post number 1208 on that thread.

Let me know if you need more screenshots. My SJQ3 is working perfectly now. I can monitor the current status. The only problems are the web UI updates every 10 seconds and that is hard coded. The other problem is a client won't work on a secondary Windows 7 computer that can't map drives to the Window XP SageTV computer. That's a problem outside of SJQ3. Many people have this problem with drive mapping, but the registry fix did not fix the problem for me.


Dave

[Slugger]

Originally Posted by wayner

My .lobs.db folder takes up over 10GB of space on my Sage server - I am guessing that this is a problem. How do I fix this?

http://code.google.com/p/sagetv-addons/wiki/H2LobsCache

Follow that doc and it'll tell you how to fix it up.

[wayner]

Will do, by the way I do have the h2-1.2.145.jar file in my JARs folder. Hopefully that means that these folders will not regenerate.

[Slugger]

Tonight's Groovy addition is keep_stb_alive.groovy.

Inspired by shadeblue's STBKeepAlive plugin and the desire to do some IR testing on a Linux Sage server, this little Groovy script will allow you to issue an IR command to your SageTV configured STBs. Unlike the STBKeepAlive plugin, this script supports any IR device configured to work in Sage and will work on any Sage supported OS. In other words, if your IR commands are working in Sage then this script will happily issue the same IR commands (since it uses the Sage API to trigger the IR commands).

If you needed to periodically send an IR command to your STB(s) then simply setup this script to run periodically via the SJQv4 crontab. The script could be easily modified to only issue the IR command if the tuner attached to the remote is not currently recording or is about to be used, etc. Such enhancements are left as an exercise for the reader.

Happy scripting!

[wayner]

I remember asking you years ago with SJQ v1 (or v2) about something like this! The problem is that the cable boxes from Rogers for SD cable, at least the SA3200, only has a power toggle, not discrete power on and power off. Fortunately the HD box has a setting that will power on when any number key is pressed.

This problem means that if for some reason the cable box has lost power that I will not get any recordings. But there is no "Power ON" command to send and sending a power toggle when the power is already on is bad news.

Any ideas on how to determine whether the power is on or off? Could you just make a 5 minute timed recording and look at the file size - will the file size be really small if the picture is just solid black? If not could you run it through Comskip and maybe you could use Comskip to tell you if it was a valid file?

One other question - does this script look to see if the tuner is currently in use? On at least one of my cable boxes I occasionally will lose audio if the box is kept on the same channel for a few days. This often happens on my SD cable box as the only channel I record on that tuner is ususally TreehouseTV - a kids' channel. Therefore I would like to change the channel and then change it back. But you don't want to do this is the tuner is in use. Also note that tuning to the current channel does not fix this problem as the box is smart enough to not retune the channel if you key in the currently watched channel.

[Slugger]

Originally Posted by wayner

I remember asking you years ago with SJQ v1 (or v2) about something like this! The problem is that the cable boxes from Rogers for SD cable, at least the SA3200, only has a power toggle, not discrete power on and power off. Fortunately the HD box has a setting that will power on when any number key is pressed.

This problem means that if for some reason the cable box has lost power that I will not get any recordings. But there is no "Power ON" command to send and sending a power toggle when the power is already on is bad news.

Any ideas on how to determine whether the power is on or off? Could you just make a 5 minute timed recording and look at the file size - will the file size be really small if the picture is just solid black? If not could you run it through Comskip and maybe you could use Comskip to tell you if it was a valid file?

Maybe, maybe, and maybe. All of this is beyond my expertise and, more importantly, my interests. Though an interesting problem, the processing of video in realtime to answer this question doesn't spark any interest in me at all. Sorry. If I were you, I'd be asking Rogers to port the "power on any key" functionality to the SD box. Having been a Rogers customer in the past, I know that is almost a waste of time.

One other question - does this script look to see if the tuner is currently in use? On at least one of my cable boxes I occasionally will lose audio if the box is kept on the same channel for a few days. This often happens on my SD cable box as the only channel I record on that tuner is ususally TreehouseTV - a kids' channel. Therefore I would like to change the channel and then change it back. But you don't want to do this is the tuner is in use. Also note that tuning to the current channel does not fix this problem as the box is smart enough to not retune the channel if you key in the currently watched channel.

As I said in the last sentence of the previous post, the script as it currently stands in svn does not check if the tuner attached to the stb is in use. It is an enhancement that is left as an exercise to the reader. That's because I just needed the script to fiddle around in Linux with my test server on some stuff and I don't actually use it on production (other than I did test it to ensure it works on Windows). I'm with Shaw Direct and my receivers don't sleep, don't need to be kept alive, etc. As I said, I just needed a quick and easy way to trigger IR commands in Linux and this was it. Figured I'd release the base functionality to svn in case others find it useful. If you wanted to make sure the tuner wasn't in use, it wouldn't be too hard. Just find the tuner device using the named remote and then ask the Sage API if it's currently recording something.

[tmiranda]

I've started working on a new Menu for SJQ4 that I've called "Task Overview". The purpose is to provide more of the information that was previously available in the SJQ3 Web interface. Attached is a screenshot that I'd like to hear opinions on.

Selecting an item on the top line filters tasks according to their state.

Selecting an item from the second line sorts on that item. Selecting the same item again reverses the sort order.

It's not ready for release yet but I wanted to get some feedback before putting a lot of effort into it.

Tom

[davephan]

The status screen would be a big improvement to SJQ4. The lack of a status screen caused me to switch back to SJQ3, since without the SJQ status screen, it is like running blind, it's really hard to figure out what is going on and if jobs are really getting processed.

The status screen looks good. I did not see is an overall number of jobs in the queue. Sometimes there are thousands of jobs in the queue if I want to process video files that were never processed with Comskip. Sometimes it can take days to process the batch of files, even with a fast quad-core processing four files at a time.

The SJQ3 web GUI has a refresh rate that is hard-coded to 10 seconds. It would be nice if the refresh rate was adjustable.

It would also be nice if the job priority for selected jobs could be increased. Maybe multiple jobs could be selected to increase or decrease the priority so the priority would not have to be changed one job at a time.

CPU priority might be another item. Although the CPU priority could be done by going to the Task Manager, select the task, and hover the mouse over the 'change priority' option in Task Manager. Maybe CPU priority would not be needed once it was determined that all the jobs were set to 'Low' CPU priority. If the 'comskip.exe' process isn't set to 'Low' priority, it can kill the computer's overall performance.

The video file type, mpeg2, TS would be nice to know. The TS files take much longer to process with Comskip than mpeg2 files. The user will then have a better idea why some jobs are taking longer to process than others in the status screen.


Dave