Plugin: Sage Job Queue (SJQ)

[nyplayer]

[QUOTESnapshot 120 adds a new server setting: IgnoreFailedTasks[/QUOTE]

Is this setup on individual tasks or is it setup on the server like priority etc..

see ex. below

if [IsTV == "true" && IsScheduledRecording == "true" && IsActivelyRecording == "false" && Filename =$ ".mpg" && FileExists != "%d%\\%p%.working" && FileExists == "%d%\\%f%.xml"]{
    :IgnoreFailedTasks true
    :PRIORITY 11
      _UNMANUALREC
      QUICKFIX
}

[Slugger]

It's a server setting... load the server settings panel from the GUI then flip the switch for the setting. The setting applies to all tasks.

[Slugger]

Originally Posted by nyplayer

Snapshot 120 adds a new server setting: IgnoreFailedTasks

Is this setup on individual tasks or is it setup on the server like priority etc..

see ex. below

if [IsTV == "true" && IsScheduledRecording == "true" && IsActivelyRecording == "false" && Filename =$ ".mpg" && FileExists != "%d%\\%p%.working" && FileExists == "%d%\\%f%.xml"]{
    :IgnoreFailedTasks true
    :PRIORITY 11
      _UNMANUALREC
      QUICKFIX
}

Just noticed your example... you should get an error from the ruleset parser with this block. Why? Because blocks that contain internal tasks may only contain one single internal task and no others. The reason for the limitation is documented here. Read the section entitled, "Limitations". Currently, the limitation is applied to all internal task ids, but after some thinking I could probably change that to just restricting the _DELETE task. The reason for the limitation is that the order of the ids you list is not necessarily the order in which they'll run since the internal task client may poll the server before your externally running client. So if you had:

if [Title =% ".*"] { # Match anything
   MYTASK
   _DELETE
}

There is no guarantee that the media object won't be deleted before MYTASK is run, which is very likely to be very bad. Again, I check for any internal task, but I can probably limit this to just restricting the _DELETE task since the rest of the internal tasks are just ones that flip flags in wiz.bin so the order in which they run shouldn't really matter. Will have to carefully consider this before switching it, however. Can anyone think of a reason where the order of the other internal tasks (other than _DELETE) would be critical?

[nyplayer]

Slugger,'
Yeah I did get an error with the new snapshot... no problem I will just seperate the 2.

[phelme]

First, thanks for the wonderful work. SJQ is great!

I'm using SJQ 2.0.0.118 & SJQC 2.0.0.112

I've got a lot of jumbled thoughts here so stay with me. I'm only trying to use SJQ 2.0 for Comskip duties at the moment, I came from 1.5 which worked great. What seems to happen to me a lot is the web interface gets "stuck" resulting in "a request timeout has expired after 5000 ms" errors when trying to log back in to the interface. This also results in the executing process queue to stop as well. And try as I might to touch the SJQ context file, resetting the date, it never recovers and I have to restart the Sage service to get it working again. In fact after a while, Jetty is just giving me 404 errors and won't even show the SJQ page at all. I'm also having problems with the completed task queue never having anything show up in it. I have tried tossing the sjqlite database, starting over and it works for a while and then the same thing happens.

what I'm wondering is if it's possible to hose everything by not waiting for various web interface tasks to complete, say by getting impatient for the server log page to finish and clicking away from it? I have anecdotal evidence that this is the case. Thoughts?

Oh BTW, sometime when I use IE6 as a web interface, the Server settings page gets an error and is blank. I usually use Firefox 3 anyway (or is Firefox still a problem, even with SJQ 2.0?).

thanks again for a great tool!

[phelme]

where does SJQ pull the "Completed Tasks" from? Now I see that there are two entries in there from last week, but even though other tasks complete, this list never changes.

[Slugger]

Originally Posted by phelme

First, thanks for the wonderful work. SJQ is great!

I'm using SJQ 2.0.0.118 & SJQC 2.0.0.112

I've got a lot of jumbled thoughts here so stay with me. I'm only trying to use SJQ 2.0 for Comskip duties at the moment, I came from 1.5 which worked great. What seems to happen to me a lot is the web interface gets "stuck" resulting in "a request timeout has expired after 5000 ms" errors when trying to log back in to the interface. This also results in the executing process queue to stop as well. And try as I might to touch the SJQ context file, resetting the date, it never recovers and I have to restart the Sage service to get it working again. In fact after a while, Jetty is just giving me 404 errors and won't even show the SJQ page at all. I'm also having problems with the completed task queue never having anything show up in it. I have tried tossing the sjqlite database, starting over and it works for a while and then the same thing happens.

One of the snapshots introduced a deadlock in the server, which would result in the timeouts as you describe. However, I quickly deleted the snapshot once I realized the bug and I don't believe anyone got a chance to download it. Certainly not an issue in 118. When these timeouts occur are you connecting from a laptop or is the browser on the same host as the one running SJQ? If the former, then the errors tend to suggest possible firewall problems preventing connections coming into Jetty/SJQ.

If you enable debugging in SageTV do you get any errors from Jetty when you start seeing these problems? What you describe sounds like a possible firewall issue on the machine running Jetty/SJQ. Do you have other apps installed in Jetty? When this happens do you get the same errors from the other apps or do those apps load fine while SJQ has these problems?

After restarting Sage, does the completed task list populate as expected or do all the tasks that had completed go back into the pending/active queue to be redone?

what I'm wondering is if it's possible to hose everything by not waiting for various web interface tasks to complete, say by getting impatient for the server log page to finish and clicking away from it? I have anecdotal evidence that this is the case. Thoughts?

I'm not crazy enough to give an outright "no" on this, but it seems like I'd have seen it happen myself during my daily usage. I can't think of why this would happen, but if you can reproduce the scenario then I'd love to get my hands on your sqlite db file for examination along with the details on how to reproduce the problem(s).

Oh BTW, sometime when I use IE6 as a web interface, the Server settings page gets an error and is blank. I usually use Firefox 3 anyway (or is Firefox still a problem, even with SJQ 2.0?).

I've had other users report some issues with IE. Personally, I haven't been able to reproduce any of them with IE6, IE7, FF2 nor FF3. If you can give me exact directions on what to do (preferably from a fresh install, empty db file) and what error I should be seeing (along with IE version) then please open an issue ticket on the project web site with those details.

[Slugger]

Originally Posted by phelme

where does SJQ pull the "Completed Tasks" from? Now I see that there are two entries in there from last week, but even though other tasks complete, this list never changes.

When the task client completes a task it reports it back to the server. The server then adds a row to the recordings table in the database. Based on the return code received from the client, that row is marked as success or failure. That flag determines which grid the row will show up in within the GUI (completed or failed).

[phelme]

Originally Posted by Slugger

One of the snapshots introduced a deadlock in the server, which would result in the timeouts as you describe. However, I quickly deleted the snapshot once I realized the bug and I don't believe anyone got a chance to download it. Certainly not an issue in 118. When these timeouts occur are you connecting from a laptop or is the browser on the same host as the one running SJQ? If the former, then the errors tend to suggest possible firewall problems preventing connections coming into Jetty/SJQ.

If you enable debugging in SageTV do you get any errors from Jetty when you start seeing these problems? What you describe sounds like a possible firewall issue on the machine running Jetty/SJQ. Do you have other apps installed in Jetty? When this happens do you get the same errors from the other apps or do those apps load fine while SJQ has these problems?

After restarting Sage, does the completed task list populate as expected or do all the tasks that had completed go back into the pending/active queue to be redone?

The browsing is done all local to the Sage server, not from a separate machine. I do have the Windows XP basic firewall enabled though but I don't think it's the problem here.

I don't get any errors in the SageTV logs that I can see, the SJQ tasks simply stop showing up there when it gets stuck.

The completed task list never populates beyond two entries that have been there since 1.5. In fact, now that I think about it, it never updated in 1.5 either beyond those two. Even when I delete the sjqlite db, the same thing happens and those two entries always show up. And here they are:

2811553 NUMB3RS/Blowback COMSKIPMPEG Nov 25, 2008 12:36:39 AM Nov 25, 2008 1:42:14 AM 127.0.0.1
274577 America's Next Top Model/Ready for My Close Up COMSKIPMPEG Nov 25, 2008 12:06:26 AM Nov 25, 2008 12:10:13 AM 127.0.0.1

Oh and nothing ever shows up in the client log either.

I think I'll maybe delete those two video files and reset the db again.

[Slugger]

Originally Posted by phelme

The browsing is done all local to the Sage server, not from a separate machine. I do have the Windows XP basic firewall enabled though but I don't think it's the problem here.

I don't get any errors in the SageTV logs that I can see, the SJQ tasks simply stop showing up there when it gets stuck.

The completed task list never populates beyond two entries that have been there since 1.5. In fact, now that I think about it, it never updated in 1.5 either beyond those two. Even when I delete the sjqlite db, the same thing happens and those two entries always show up. And here they are:

2811553 NUMB3RS/Blowback COMSKIPMPEG Nov 25, 2008 12:36:39 AM Nov 25, 2008 1:42:14 AM 127.0.0.1
274577 America's Next Top Model/Ready for My Close Up COMSKIPMPEG Nov 25, 2008 12:06:26 AM Nov 25, 2008 12:10:13 AM 127.0.0.1

Oh and nothing ever shows up in the client log either.

I think I'll maybe delete those two video files and reset the db again.

What's your ruleset look like?

[phelme]

Originally Posted by Slugger

What's your ruleset look like?

server:

// Comskip MPEG scan
if [IsTV == "true" && IsActivelyRecording == "false" && FileExists == "%d%\\%p%.mpg" && FileExists != "%d%\\%p%.edl" && ChannelNumber != "0" && ChannelNumber != "1"]
{
COMSKIPMPEG
}

// Comskip TS scan
if [IsTV == "true" && IsActivelyRecording == "false" && IsWatched == "false" && FileExists == "%d%\\%p%.ts" && FileExists != "%d%\\%p%.edl" && ChannelNumber != "0" && ChannelNumber != "1"]
{
COMSKIPTS
}


client:

:MAXPROCS 3 # Client can run three jobs in parallel, but only max of 2 comm flag jobs

# Exe block for COMSKIP_MPEG jobs (must match tag from server ruleset)
COMSKIPMPEG {
:CPU LOW
:MAX 2
"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\comskip80_006\\comskip.exe -n \"%c%\""
}

# Exe block for COMSKIPTS jobs (must match tag from server ruleset)
COMSKIPTS {
:CPU LOW
:MAX 2
"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\comskip80_006\\comskip.exe -n \"%c%\""
}

---
I split the Comskip duties because I'm testing out the new H.264 version and want at least the mpeg files to complete. the beta Comskip sometimes crashes on the .ts files from the HDPVR.

Right now things are running smoothly and several comskip tasks have run just fine, but again the Completed Task list never updates.

[Slugger]

Originally Posted by phelme

server:

// Comskip MPEG scan
if [IsTV == "true" && IsActivelyRecording == "false" && FileExists == "%d%\\%p%.mpg" && FileExists != "%d%\\%p%.edl" && ChannelNumber != "0" && ChannelNumber != "1"]
{
COMSKIPMPEG
}

// Comskip TS scan
if [IsTV == "true" && IsActivelyRecording == "false" && IsWatched == "false" && FileExists == "%d%\\%p%.ts" && FileExists != "%d%\\%p%.edl" && ChannelNumber != "0" && ChannelNumber != "1"]
{
COMSKIPTS
}


client:

:MAXPROCS 3 # Client can run three jobs in parallel, but only max of 2 comm flag jobs

# Exe block for COMSKIP_MPEG jobs (must match tag from server ruleset)
COMSKIPMPEG {
:CPU LOW
:MAX 2
"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\comskip80_006\\comskip.exe -n \"%c%\""
}

# Exe block for COMSKIPTS jobs (must match tag from server ruleset)
COMSKIPTS {
:CPU LOW
:MAX 2
"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\comskip80_006\\comskip.exe -n \"%c%\""
}

---
I split the Comskip duties because I'm testing out the new H.264 version and want at least the mpeg files to complete. the beta Comskip sometimes crashes on the .ts files from the HDPVR.

Right now things are running smoothly and several comskip tasks have run just fine, but again the Completed Task list never updates.

Ok, I think I know the answer...

comskip.exe returns 1 on success and some other number on failure. I believe I've dealt with this issue before with someone else (don't have time to go reading back through this thread right now, however).

So what's happening is that the client tells the server that comskip failed each time because it's returning 1, which is why it isn't ending up in the completed list. Each run should end up in the failed list. Are they not moving to the failed list? But since you check for the edl file in your test then the jobs will never be reran since the edl is actually being created (since comskip is actually succeeding even though it's tell the caller [sjqc] that it failed).

I use ShowAnalyzer myself, but I'm having a deja vu moment as I write this. The best solution is to have the author(s) of comskip return 0 on success, which is the standard practice, or assuming that won't get done then call comskip from a batch file that returns 0 when comskip returns 1 and 1 when comskip does not return 1.

So now I'm curious as to why the jobs aren't being added to the failed list or are they?

[phelme]

Originally Posted by Slugger

Ok, I think I know the answer...

comskip.exe returns 1 on success and some other number on failure. I believe I've dealt with this issue before with someone else (don't have time to go reading back through this thread right now, however).
...
So now I'm curious as to why the jobs aren't being added to the failed list or are they?

yeah, they are.

I guess I hadn't checked since I moved back to Firefox from IE6. I think I was having some of the same problems I was having with server settings not being displayed. I just checked and there were a bunch of items in the failed list, even though they aren't failing.

---

I'll talk to the comskip author and see if there is anything that can be done about the return code. Thanks!!

[nyplayer]

Phelme,

Use something like this

comskip.bat
cd c:\comskip
c:
start /b /w /belownormal comskip.exe -n "%~f1"
set comskiperror=%ERRORLEVEL%
if %comskiperror% EQU 1 exit 0
if %comskiperror% NEQ 1 exit 1

Originally Posted by phelme

yeah, they are.

I guess I hadn't checked since I moved back to Firefox from IE6. I think I was having some of the same problems I was having with server settings not being displayed. I just checked and there were a bunch of items in the failed list, even though they aren't failing.

---

I'll talk to the comskip author and see if there is anything that can be done about the return code. Thanks!!

[phelme]

Originally Posted by nyplayer

Phelme,

Use something like this

comskip.bat
cd c:\comskip
c:
start /b /w /belownormal comskip.exe -n "%~f1"
set comskiperror=%ERRORLEVEL%
if %comskiperror% EQU 1 exit 0
if %comskiperror% NEQ 1 exit 1

thanks for code. I actually wrote some myself, but I keep getting errors like this when calling the batch file:

Thu Nov 27 10:07:45 PST 2008: Starting task with priority 1/1/10
Executing command line: C:\Program Files\SageTV\SageTV\ComskipMonitor\test.bat "\\sagetv-server\BigMedia1-Recordings\Californication-LaRonde-3145267-0.mpg"
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
---
if I run the batch file solo, i.e. no %c%, it does execute. there is some oddness going on with either environmental variables or spaces. what is odd is I've been running the same path with comskip directly from SJQ, like this:

"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\comskip80_006_donators\\comskip.exe -n \"%c%\""

with no problems. but for some reason this:

"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\test.bat \"%c%\""

doesn't work. I may just move the comskip directory directly under c:\ to avoid spaces and see if that fixes it. Thanks.

[Slugger]

Originally Posted by phelme

thanks for code. I actually wrote some myself, but I keep getting errors like this when calling the batch file:

Thu Nov 27 10:07:45 PST 2008: Starting task with priority 1/1/10
Executing command line: C:\Program Files\SageTV\SageTV\ComskipMonitor\test.bat "\\sagetv-server\BigMedia1-Recordings\Californication-LaRonde-3145267-0.mpg"
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
---
if I run the batch file solo, i.e. no %c%, it does execute. there is some oddness going on with either environmental variables or spaces. what is odd is I've been running the same path with comskip directly from SJQ, like this:

"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\comskip80_006_donators\\comskip.exe -n \"%c%\""

with no problems. but for some reason this:

"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\test.bat \"%c%\""

doesn't work. I may just move the comskip directory directly under c:\ to avoid spaces and see if that fixes it. Thanks.

Try:

"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\test.bat" "\"%c%\""

[phelme]

Originally Posted by Slugger

Try:

"C:\\Program Files\\SageTV\\SageTV\\ComskipMonitor\\test.bat" "\"%c%\""

yeah, that's not working either. I'll try some more quote and escaped quote combinations. It has something to do with the spaces in the path.

[nyplayer]

Phelme,

The first thing you should do is make sure yor test.bat file works by dragging an mpg onto it and see if comskip runs ok.

If that doesn't work then post your test.bat.

[phelme]

Originally Posted by nyplayer

Phelme,

The first thing you should do is make sure yor test.bat file works by dragging an mpg onto it and see if comskip runs ok.

If that doesn't work then post your test.bat.

yeah, both batch scripts (the one I wrote and the one above) do. I've given up trying different combinations of escaping every quote, space etc. in the SJQ client and simply moved the comskip folder directly under c:\. now it works fine with:

"C:\\ComskipMonitor\\test.bat" "%c%"

whatever.

[Slugger]

I'm getting close to making the first "official" release of SJQ 2. New builds of the client and server are ready for testing. Key updates:

• Parse and handle raw numbers in the ruleset file better; should get fewer errors from ruleset parser when dealing with numbers
• Rearrange status bar; make it more compact for 1024x768 users
• Add file cleaner back to core (a.k.a. the "edl cleaner")
• Add timeout option for tasks

The timeout option allows you to specify a maximum run time for a task and if that run time expires then the task will immediately be killed and marked as failed. You can specify a static value or a variable value that depends on the length of the media object file. Details on how to use this feature are documented in the wiki doc for the client configuration.

The server protocol was increased to handle the new timeout options. Therefore server build 135 requires client build 136 or higher. The protocol grid is available here. Use the grid to match clients and servers together so that you are running compatible versions of the client and server.

There is one big item left on the todo list before I'm ready to call this stable/feature complete for 2.0.0. However, if there's something missing that you want to see then this would be the time to speak up. Once I release 2.0.0 (within the next week or so) then SJQ will definitely be going into maintenance mode with only major bug fixes being addressed. No new development work will be done probably until the spring of 2009 (plans for my next plugin project are underway and once I get going on it, it will probably consume all of my SageTV time over the next few months).