Plugin: SJQv4

[Slugger]

Need to see the error message(s). I can't even try to guess which method call it's complaining about.

[Slugger]

Add in the output of the println showing the command array, too

[bikesquid]

Here's the log printout, with the error there's no println, I got that before I started adding the path before the target.
error message.png

[Slugger]

Ok, I'll admit that error requires a little more "programmer" experience.

The problem it's trying to tell you about is that a variable of type File does not support the '+' operator. In other words, this line is invalid:

def fullpathfile = (newDir + "\\" + newName + ".ts")

And that's because newDir and/or newName is of type File and you can't concatenate ("add") strings together with a File instance. To solve, try this:

def fullpathfile = newDir.toString() + "\\" + newName + ".ts"

I'm assuming newName is already a String, if not just add a .toString() to it as well. I remove the parenthesis because they add no value to the statement and aren't necessary (though the use of them is not invalid, but discouraged as it makes the code a little more difficult to read).

One thing to learn here is that all objects in Java (and therefore Groovy) have a toString() method you can call on them to covert the object to a String. But be careful, because not all classes implement this as you'd expect (File objects, however, do so you're fine here).

The other lesson. The more you play with Groovy scripting, the more you'll want to try and become familiar with the error messages and what they're trying to tell you. Now, as I say, I don't think this one would be clear to the average user so I'm not saying you should have got this one. What I am saying, is now you've seen it, I've explained its meaning and hopefully the next time you run into it you'll recognize it.

[bikesquid]

Slugger - Seriously, thank you. I know it's frustrating with us dim folks trying to program. Honestly, I search for the error messages, and I find the answers, but the answers make no damn sense. The problem for me is I don't have a base from which to learn. Doing my best to solve my own problems but too often I just have to say "huh?" and beg for a push. I've been on the verge of giving up and just calling a batch file and doing it from there, but it seems it'd be worth struggling a bit for the result. In case anyone can springboard from this point, here's the code so far, not elegant but seems to be doing the rename reliably and starting the comskip. Next is to add some error checking and unique filename logic, etc. But that'll be a ways down the road for me. Off to drink some single malt and watch TV for a change!!!
Thanks again.

import org.apache.commons.io.FilenameUtils
import org.apache.commons.io.FileUtils

private class Settings {
    static public final boolean TEST_MODE = false
}

def mf = MediaFileAPI.GetMediaFileForID(SJQ4_METADATA["SJQ4_ID"].toInteger())
if(mf == null) {
    println "Invalid media file id! [${SJQ4_METADATA['SJQ4_ID']}]"
    return 1
}

AiringAPI.SetWatched(mf); // Lets set the show as watched before we go further


def title = ShowAPI.GetShowTitle(mf)
def sNum = ShowAPI.GetShowSeasonNumber(mf)
def eNum = ShowAPI.GetShowEpisodeNumber(mf)
def seNum = sNum > 0 && eNum > 0 ? String.format("S%02dE%02d", sNum, eNum) : null
def subtitle = ShowAPI.GetShowEpisode(mf)
def origAirDate = ShowAPI.GetOriginalAiringDate(mf) > 0 ? new Date(ShowAPI.GetOriginalAiringDate(mf)).format("yyyy-MM-dd") : null
def newPrefix = "${title} - "
if(seNum != null)
    newPrefix += seNum
else if(subtitle != "")
    newPrefix += subtitle
else if(origAirDate != null)
    newPrefix += origAirDate
else
    newPrefix += "UNKNOWN"

def numSegments = MediaFileAPI.GetNumberOfSegments(mf)
if(numSegments == 1) {
    def prefix = FilenameUtils.getBaseName(MediaFileAPI.GetFileForSegment(mf, 0).getAbsolutePath())
    println "Renaming files that look like '${prefix}.*' to '${newPrefix}.*'..."
    renameMatches(MediaFileAPI.GetParentDirectory(mf), prefix, null, newPrefix)
} else if(numSegments > 1) {
    for(def i = 0; i < numSegments; ++i) {
        def prefix = FilenameUtils.getBaseName(MediaFileAPI.GetFileForSegment(mf, i).getAbsolutePath())
        def segPrefix = "$newPrefix-$i"
        println "Renaming files that look like '${prefix}.*' to '${segPrefix}.*'..."
        renameMatches(MediaFileAPI.GetParentDirecotry(mf), prefix, null, segPrefix)
    }
} else {
   println "No file segments for given media file!"
   return 1
}
return 0

// Pass null for newDir to keep renamed file in same directory
def renameMatches(def oldDir, def prefix, def newDir, def newPrefix) {
    if(newDir == null)
        newDir = oldDir
    Utility.DirectoryListing(oldDir).each {
        if(FilenameUtils.wildcardMatchOnSystem(it.getName(), "${prefix}.*")) {
            def newName = (newPrefix + it.getName().substring(it.getName().indexOf('.')) =~ /[\/\\:*?<>]/).replaceAll("")
            if(!Settings.TEST_MODE) {
                FileUtils.moveFile(it, new File(newDir, newName))
            } else {
                println "Would rename '$it' to '$newName' if test mode were disabled!"
            }
        }
    }
// comskip attempt

def newName2 = (newPrefix =~ /[\/\\:*?<>]/).replaceAll("")
def fullpathfile = newDir.toString() + "/" + newName2 + ".ts"

def command = ["c:/program files/comskip/comskip.exe", fullpathfile]

println "full command will be '$command'"
def proc = command.execute()
if(!Settings.TEST_MODE) {
          def initialSize = 4096
          def outStream = new ByteArrayOutputStream(initialSize)
          def errStream = new ByteArrayOutputStream(initialSize)

              proc.consumeProcessOutput(outStream, errStream)
              proc.waitFor()
              println 'out:\n' + outStream
              println 'err:\n' + errStream

            } else {
                println "Would run '$command' if test mode were disabled!"
            }
}

return 0;

[Slugger]

A few things I'll mention:

1) Keep plugging away at it. I don't mean to put down Windows batch programming because I'm sure some people swear by it and those people can probably do a lot of crazy impressive things with it, but I'll take Groovy (or Perl or PHP or anything) over batch files any day. Try to write an equivalent batch file that's doing what your Groovy script is doing... not fun, if at all possible.

2) Ask questions, I don't mind, but the replies won't always show up 3 minutes after you post. A late night of work, hence the quick replies.

3) If you're willing to put in the time and you're able to pick this up, there is absolutely nothing you can't do with Groovy. This is why I moved to a full, proper scripting platform. Yes, there's lots more to learn, but because there's lots more to learn there's way, way more you can do! Once you get the simple stuff down, then you can go crazy. For example, I've been working on adding a BitTorrent "recorder" to supplement my satellite inputs to grab content not available to me on TV and to download missed recordings caused by conflicts, failed recordings or identified bad recordings. Completely written in Groovy using SJQv4 as the task executor. Try to do that with a batch file. Certainly could never be done with SJQv3 (at least not in any way that would maintain my sanity). In SJQv4: It's 4 or 5 Groovy scripts, none longer than 120 lines (I think). By moving to a proper scripting platform, I'm not maintaining and fixing parser code and so forth, instead I've got a base platform built on Groovy and now I'm really starting to exploit it in ways I've never imagined. My end goal with SJQv4 is that others start to exploit this platform in a similar fashion, coming up with enhancement ideas to add new features to SageTV that no one's thought of yet.

[bikesquid]

Off the wall question - Is there a way for me to drop a file onto 'something' and have it join the sjq queue? I like to review the comskip results before cutting and remuxing but I'd like to drop those tasks back in sjq so it can prioritize and manage the task queue as defined. ... The script I imagine is the same groovy script, but the hook to sjq is maybe more what I'm asking about... though I could be wrong.

[Slugger]

Originally Posted by bikesquid

Off the wall question - Is there a way for me to drop a file onto 'something' and have it join the sjq queue? I like to review the comskip results before cutting and remuxing but I'd like to drop those tasks back in sjq so it can prioritize and manage the task queue as defined. ... The script I imagine is the same groovy script, but the hook to sjq is maybe more what I'm asking about... though I could be wrong.

Sure... you'd just have to write a Groovy script that accepted the file name as a command line arg and knew what to do with it. In this case, it'd be to find the media file associated with whatever you dragged onto the script, then queue that media file up in the SJQ queue.

EDIT: You would probably use SageGroovy as the engine to execute the Groovy scripts from the Windows desktop.

[K O]

How do I turn off the .lobs.db logging altogether? I keep filling up the hard drive will all these null pointer exceptions... hundreds of files with the error which I'm told is harmless, however the c:\ drive on my Windows Home Server is only 20GB and it fills it up pretty fast with these logs. It stops when I disable the SJQ4 plugin.




==============================

===== Tue Mar 29 17:06:33 CDT 2011 =====

LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
javax.script.ScriptException: java.lang.NullPointerException


==============================

===== Tue Mar 29 17:07:03 CDT 2011 =====

LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties

[Slugger]

Originally Posted by K O

How do I turn off the .lobs.db logging altogether? I keep filling up the hard drive will all these null pointer exceptions... hundreds of files with the error which I'm told is harmless, however the c:\ drive on my Windows Home Server is only 20GB and it fills it up pretty fast with these logs. It stops when I disable the SJQ4 plugin.

You can't, it's a function of the H2 database client. It's just the way it works, albeit buggy. Check to see if you're using the h2-1.2.145.jar (in your JARs) folder. I've had pretty decent luck with this version not filling up the cache dir, though it's not perfect. If you're on an older version, update the the sagex-h2 plugin in SageTV.

==============================

===== Tue Mar 29 17:06:33 CDT 2011 =====

LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties
javax.script.ScriptException: java.lang.NullPointerException


==============================

===== Tue Mar 29 17:07:03 CDT 2011 =====

LOG4J: Configured Logging for: sagex-api using file: sagex-api.log4j.properties

This is a misconfiguration with one of your tasks. I don't think I've ever said these errors were harmless. If I did, then I misspoke. I mean they aren't harmless in that they cause damage, but it's a clear sign of a buggy Groovy script and/or a misconfigured task. Find out which task(s) are producing that output and fix it. Look at the task client log files for the time stamp given (probably a second or two earlier, actually) and then figure out which task is running at that time and then fix it so it doesn't constantly fail with a NullPointerException. Depending on what script is causing it, an NPE failure may send the the task right back to the queue and the cycle will continue forever. Scripts that cause NPEs are either buggy or misconfigured, either way something needs to be fixed, but before I can even try to diagnose the problem, I need to know what script is causing that output.

[bikesquid]

Question: I'm trying to determine if a recording is a movie or series. In the past I used the value of "MediaType=" from the .properties (value would be either Movie or TV) but that's not always available to me now. Is there a simple test I can do using the sjq4_metadata or MediaFileAPI? I've found "IsTVFile " but have no idea if that's the right answer. MediaType within SJQ seems to be different as it reflects a recording is in fact a media file....

Second question: If it is a movie will MediaFileAPI.GetShowYear(mf) return the production year rather than current airing?

[Slugger]

Originally Posted by bikesquid

Question: I'm trying to determine if a recording is a movie or series. In the past I used the value of "MediaType=" from the .properties (value would be either Movie or TV) but that's not always available to me now. Is there a simple test I can do using the sjq4_metadata or MediaFileAPI? I've found "IsTVFile " but have no idea if that's the right answer. MediaType within SJQ seems to be different as it reflects a recording is in fact a media file....

Second question: If it is a movie will MediaFileAPI.GetShowYear(mf) return the production year rather than current airing?

IsTVFile() returns true if the media file was recorded (as opposed to imported); movies recorded from tv would also return true on this call. Imported tv shows would return FALSE for this call, unless you linked them to an EPG airing after importing them.

If I were checking if a tv recording was a movie, I'd probably use this:

def cats = ShowAPI.GetShowCategoriesList(mf)
if(cats.contains("Movie") || cats.contains("Film")) {
   // It's a movie, do your thing for a movie here...
} else {
   // It's not a movie, do your thing for a series here...
}

As for GetShowYear(), according to the javadocs, it's not quite clear what this value represents. If I had to guess, I'd say, yes, it returns the year of release for movies, but that's only a guess. You may want to post a message in the Studio forum for clarification from those in the know.

[bikesquid]

Is there an 'easy' way to incrimentally +1 to filename and retest till it doesn't already exist? i.e. if Dexter - S04E03.ts already exists try Dexter - S04E03-1.ts, Dexter S04E03-2.ts, etc till we've found a 'safe name' to proceed with....

I'm planning to use something like this but don't know how to loop or add 1...

if(fullpathfile.exists()) {
println($fullpathfile" already exists for this recording!");
fullpathfile = fullpathfile + " - 1"

// insert +1 loop logic here....

}

[Slugger]

We'll turn you into a programmer yet!

import org.apache.commons.io.FilenameUtils

def file = new File("C:/temp/test.txt")

def dir = file.getParent()
def base = FilenameUtils.getBaseName(file.getName())
def ext = FilenameUtils.getExtension(file.getName())
def i = 1
while(file.exists())
    file = new File(dir, "$base-${i++}.$ext")
println "New file is: $file"

Not sure how you're testing all of your code, but you may want to use an IDE to experiment with. Sure beats having to setup tasks and schedule them to run in SJQv4 to test your code. I use SageGroovy. Read up on starting GUI mode with that. You may also try Eclipse, but I'd probably stick to SageGroovy to start with. Either IDE allows you to write scripts and run them directly from the editor - makes developing/testing much, much easier.

Happy scripting!

[bikesquid]

Originally Posted by Slugger

We'll turn you into a programmer yet!

I feel sick

Originally Posted by Slugger

Not sure how you're testing all of your code, but you may want to use an IDE to experiment with. Sure beats having to setup tasks and schedule them to run in SJQv4 to test your code. I use SageGroovy. Read up on starting GUI mode with that. You may also try Eclipse, but I'd probably stick to SageGroovy to start with. Either IDE allows you to write scripts and run them directly from the editor - makes developing/testing much, much easier.

Yeah, I saw that and was starting to fiddle with it when I read that it likely wasn't happy on 64bit win7... which is all I've got, and that it wouldn't accept the SJQ4_METADATA 'stuff' so I got scared and ran away. Would love to get it working so I could sort out all the BS typos and such without the hugely annoying process of adding tasks manually to sage. Maybe I'll have a beer and give it another go...

[Slugger]

Originally Posted by bikesquid

Yeah, I saw that and was starting to fiddle with it when I read that it likely wasn't happy on 64bit win7... which is all I've got, and that it wouldn't accept the SJQ4_METADATA 'stuff' so I got scared and ran away. Would love to get it working so I could sort out all the BS typos and such without the hugely annoying process of adding tasks manually to sage. Maybe I'll have a beer and give it another go...

Give it a shot on Win64, would you? And let me know if it starts up. The IDE itself will run on Win64, just the helper exe I created to launch it may or may not. If it doesn't, you could write a batch file that launched it for you. Alternatively, Eclipse env would work on Win64, but a lot more work involved setting up an Eclipse env to run SJQv4 scripts.

As for the SJQ4_MEATDATA missing, yes, it's missing, but just create a "temp" one at the top of your script to simulate it while testing, then when you're ready for production, just remove it from the script - it's what I do.

def SJQ4_METADATA = ["SJQ4_ID":"1234566", "SJQ4_TYPE":"MediaFile"]

// And then do whatever you were doing as normal; just remember to
// remove the above line before using the scirpt in SJQv4.

[Spectrum]

It actually works quite well on Win7 x64 Slugger. I thought I told you about that awhile back but maybe I forgot to actually post it I can work on my primary dev box that just has Placeshifter on it and can run scripts against the Sage server in the other room, both machines are running Win7 x64.

It beats writing a script then queuing it up inside SJQ to test with a stick!

[Slugger]

Originally Posted by Spectrum

It actually works quite well on Win7 x64 Slugger. I thought I told you about that awhile back but maybe I forgot to actually post it I can work on my primary dev box that just has Placeshifter on it and can run scripts against the Sage server in the other room, both machines are running Win7 x64.

It beats writing a script then queuing it up inside SJQ to test with a stick!

Excellent. Thanks for the info.

[rsagetv99]

Is there a "return" code for jobs that I don't want to show in the completed queue? I would rather not fill up the completed queue with jobs that were skipped because of the wrong file-type or because an edl already exists.

[Slugger]

Originally Posted by rsagetv99

Is there a "return" code for jobs that I don't want to show in the completed queue? I would rather not fill up the completed queue with jobs that were skipped because of the wrong file-type or because an edl already exists.

I assume you're returning code 2 from the test script for these tasks? That marks them as 'skipped' in the database and by default skipped tasks purge themselves from the db after 24 hours. The amount of time tasks hang around in the db is configurable in the SJQ plugin config screen from the Sage STV. One day is the lowest you can go, and that's the default for skipped tasks. Completed tasks hang around for 7 days and failed tasks for 21 by default. Again, it's all configurable.