Comskip Monitor: Announcement

[gplasky]

Make sure the clients are using the same STV as the server that you checled playback with and that comskip was enabled on each client.

Gerry

[awhenry]

In case anyone is interested, I hacked together a multithreaded version of this app that will run multiple comskips (or showanalyzers I suppose, though I haven't tried that) at once. I have been using it for a few days with no problems, but I am not exactly a Java whiz so there could be latent problems I haven't encountered (yet). Also, I haven't tried to include the thread configuration in the web interface, so you'll have to edit your csm.properties by hand to make best use of this (it defaults to two threads if you don't edit your properties file).

Anyway, if you want to give it a try, download the modified csm.jar from the attached zip. Make a backup of your current csm.jar (found in the install directory, C:\ComskipMonitor\data for me) and then replace it with the downloaded one (you will need to stop the service before replacing this file). Finally, edit your csm.properties file (again found in the install directory, C:\ComskipMonitor in my case) to include the line "max.threads=<however many simultaneous comskip copies you want to run>" and restart the service. I use max.threads=4 myself with a Q6600, though with only a single HDHR I don't really get over 2 going at once very often.

Thanks to the original author for this tool! I installed it when I first got SageTV a few months ago and haven't looked back since. The only thing I ever wished for was the ability to mark multiple concurrent shows at once. Anyone else that wished for the same, Merry Christmas .

[hemicuda]

super. I've been running a 2nd comskip folder for the run when watched part of the STVi since CSM was a single-thread app. this will be nice when I get a chance to try it out.

thanks

[timg11]

----deleted - moved to new thread---"Comskip Monitor - reboot loop when installed?"----

[AJ Bertelson]

Originally Posted by gplasky

Make sure the clients are using the same STV as the server that you checled playback with and that comskip was enabled on each client.

Gerry

Yep all clients have comskip enabled as a global setting under sagemc and it still doesn't work.

[appelm]

Originally Posted by awhenry

In case anyone is interested, I hacked together a multithreaded version of this app that will run multiple comskips (or showanalyzers I suppose, though I haven't tried that) at once. I have been using it for a few days with no problems, but I am not exactly a Java whiz so there could be latent problems I haven't encountered (yet). Also, I haven't tried to include the thread configuration in the web interface, so you'll have to edit your csm.properties by hand to make best use of this (it defaults to two threads if you don't edit your properties file).

Anyway, if you want to give it a try, download the modified csm.jar from the attached zip. Make a backup of your current csm.jar (found in the install directory, C:\ComskipMonitor\data for me) and then replace it with the downloaded one (you will need to stop the service before replacing this file). Finally, edit your csm.properties file (again found in the install directory, C:\ComskipMonitor in my case) to include the line "max.threads=<however many simultaneous comskip copies you want to run>" and restart the service. I use max.threads=4 myself with a Q6600, though with only a single HDHR I don't really get over 2 going at once very often.

Thanks to the original author for this tool! I installed it when I first got SageTV a few months ago and haven't looked back since. The only thing I ever wished for was the ability to mark multiple concurrent shows at once. Anyone else that wished for the same, Merry Christmas .

Thanks for this.. Snap to install and working great!

[mzup]

Originally Posted by HeeZy

i have a problem here. i just installed comskip and let it run on all my files overnight.

i then change a few settings by running setup.bat, and clicking "Finish" on the last page. the service restarts only to rescan all my files again!? why?

what makes comskip decide to scan a file? or what files does comskip look for so that it doesn't scan a file it already scanned?


thanks!

I noticed the same thing here. It keeps doing the same movies. I know there's a log in the data folder, but that is not preventing it from doing the same ones over and over.
Any help please, Steep

[smdl]

Originally Posted by awhenry

In case anyone is interested, I hacked together a multithreaded version of this app that will run multiple comskips (or showanalyzers I suppose, though I haven't tried that) at once. I have been using it for a few days with no problems, but I am not exactly a Java whiz so there could be latent problems I haven't encountered (yet). Also, I haven't tried to include the thread configuration in the web interface, so you'll have to edit your csm.properties by hand to make best use of this (it defaults to two threads if you don't edit your properties file).

Anyway, if you want to give it a try, download the modified csm.jar from the attached zip. Make a backup of your current csm.jar (found in the install directory, C:\ComskipMonitor\data for me) and then replace it with the downloaded one (you will need to stop the service before replacing this file). Finally, edit your csm.properties file (again found in the install directory, C:\ComskipMonitor in my case) to include the line "max.threads=<however many simultaneous comskip copies you want to run>" and restart the service. I use max.threads=4 myself with a Q6600, though with only a single HDHR I don't really get over 2 going at once very often.

Thanks to the original author for this tool! I installed it when I first got SageTV a few months ago and haven't looked back since. The only thing I ever wished for was the ability to mark multiple concurrent shows at once. Anyone else that wished for the same, Merry Christmas .

Thanks for doing this, awhenry! I have 4 threads going right now (just installed CSM today), and it seems to be working fine.

Shaun

[psklenar]

I'm running my Sage installation on Windows Home Server ... I'm still learning all of this (WHS, Sage, Comskip, Comskip Monitor, etc) ... would the right way to change from drive letters to UNC's be to edit csm.properties to change

file.directories=D\:\\shares\\Recordings
to
file.directories=\\<servername>\Recordings
or would this be smarter ...
file.directories=\\localhost\Recordings

Thanks,
pat----

[cychou]

Originally Posted by psklenar

file.directories=\\<servername>\Recordings

Use double back slash (\\) for every single back slash (\):

file.directories=\\\\<servername>\\Recordings

[tpboyce]

It looks like WHS version of SageTV does not install java.exe. I am thinking about doing an MSI installer for comskip monitor that would also have an AddIn configuration page for WHS. Would anyone be interested in this?

[gplasky]

Originally Posted by tpboyce

It looks like WHS version of SageTV does not install java.exe. I am thinking about doing an MSI installer for comskip monitor that would also have an AddIn configuration page for WHS. Would anyone be interested in this?

That would be nice. And yes-the .msi installer for SageTV on WHS is just that-an add-in and not an all-in-one installation like the other Windows platform. So you need to make sure your prerequesites like Java are installed.

Gerry

[kemputer26]

Hi all,
Vista with UAC disabled, Sage running in service mode, Comskip 80.008, ComskipMonitor 2.7, HDPVR files...

CSM Service is set to run as an administrator, double-checked.
comskip.exe executes in Task Manager as the administrator user I selected in the service configuration. User has rights to execute locally and log on as a service, and is part of the permissions group for /program files/ and all subfolders (full control).

I can configure Comskip to run, which it does fine from command line.

Everything works great, and I've read a LOT of threads to get it that way, but I have a sticky problem:
My comskip.exe does not terminate after it completes processing when called by Comskipmonitor. I've left it alone for a whole day, it just sits at 500k of mem and 0% CPU forever.
I kill the process, and CSM immediately launches and starts processing the next file, which also works fine, right up until it's finished, then it hangs...



I've created a workaround by defining 10 execution threads (Thanks for the multithreaded CSM, awhenry, excellent work ). Now I only have to go in once per day and kill the ~9 "dead" instances of comskip.exe...but it's a kludge and not a good one.

Any ideas on why comskip.exe may not finalize and unload when it's finished processing? I don't want to write code to hunt down dead comskip.exe threads and kill them every night at midnight.

Any help would be greatly appreciated...even if you saw this problem once and have some obscure idea on what it MIGHT be...I'm out of ideas.

Cheers
Jeff

[awhenry]

Originally Posted by kemputer26

Any ideas on why comskip.exe may not finalize and unload when it's finished processing?

I'm afraid I haven't encountered this problem myself, so I don't have any specific suggestions for you. I do wonder if there are any clues at the end of the log files created during comskip's processing? The last few lines should include a summary of the number of frames processed as well as a timestamp for when the processing completed if it completed successfully.

Is there another version of comskip you could try that supports the HDPVR files, just in case there is an odd bug in the version of comskip you are using? You might also want to check that you are running the latest version of Java.

Any chance you have an MPEG2 file you could drop in your recordings directory for processing? Just to see if the issue is specific to the HDPVR files or not.


Andrew

[kemputer26]

Originally Posted by awhenry

The last few lines should include a summary of the number of frames processed as well as a timestamp for when the processing completed if it completed successfully.

53711 Frames Processed
################################################################
Time at end of run:
Wed Jan 28 20:25:16 2009
################################################################
First PAT [ 0] [ 100]
First PAT [ 0] [ 100]

Originally Posted by awhenry

Is there another version of comskip you could try that supports the HDPVR files, just in case there is an odd bug in the version of comskip you are using? You might also want to check that you are running the latest version of Java.

I'm using the latest version of comskip, a donor version (paid) and the very latest.
I have Java version 6 Update 11 (the very latest AFAIK)

Originally Posted by awhenry

Any chance you have an MPEG2 file you could drop in your recordings directory for processing? Just to see if the issue is specific to the HDPVR files or not.
Andrew

Good suggestion...I'll try that

[erik]

Originally Posted by kemputer26

53711 Frames Processed
################################################################
Time at end of run:
Wed Jan 28 20:25:16 2009
################################################################
First PAT [ 0] [ 100]
First PAT [ 0] [ 100]

The last two lines suggest Comskip is going into interactive UI mode.
Do you have the -w switch in the command line?
Do you run ComskipGUI.exe instead of Comskip.exe
Do you have
output_debugwindow=1
in your comskip.ini file?

[gplasky]

Originally Posted by erik

The last two lines suggest Comskip is going into interactive UI mode.
Do you have the -w switch in the command line?
Do you run ComskipGUI.exe instead of Comskip.exe
Do you have
output_debugwindow=1
in your comskip.ini file?

Had the same thing happen to me last week on one episode of Burn Notice. But it did not happen on other h.264 recordings like Battlestar Galactica or the Law and Order series. So I was considering it a fluke. Here's the log. Comskip runs (Dirmon2) for about 5 seconds and doesn't generate the .edl or the .Vprj file for me.

################################################################
Generated using Comskip 0.80.008
Time at start of run:
Thu Jan 29 09:12:04 2009
################################################################
Mpeg:	\\MAIN\SageTV\BurnNotice-DoNoHarm-8673938-0.ts
Exe	comskip.exe
Logo:	\\MAIN\SageTV\BurnNotice-DoNoHarm-8673938-0.logo.txt
Ini:	C:\Comskip-ts\comskip.ini

Detection Methods to be used:
	1) Black Frame
	2) Logo - Give up after 2000 seconds
	3) Resolution Change
	4) Aspect Ratio


ComSkip throttles back from -0001 to -0001.
The time is now 0912 so it's full speed ahead!

Settings
--------
[Main Settings]
;the sum of the values for which kind of frames comskip will consider as possible cutpoints: 1=uniform (black or any other color) frame, 2=logo, 4=scene change, 8=resolution change, 16=closed captions, 32=aspect ration, 64=silence, 255=all.
detect_method=43
;Set to 10 to show a lot of extra info, level 5 is also OK, set to 0 to disable
verbose=10
;Frame not black if any of the pixels of the frame has a brightness greater than this (scale 0 to 255)
max_brightness=60
maxbright=1
;Frame not pure black if a small number of the pixels of the frame has a brightness greater than this. To decide if the frame is truly black, comskip will also check average brightness (scale 0 to 255)
test_brightness=40
;
max_avg_brightness=25
;
max_commercialbreak=600
;
min_commercialbreak=25
;
max_commercial_size=125
;
min_commercial_size=4
;
min_show_segment_length=125
;
max_volume=500
;
non_uniformity=500
[Detailed Settings]
min_silence=12
noise_level=5
brightness_jump=200
fps=25
validate_silence=1
validate_uniform=1
validate_scenechange=1
global_threshold=1.05
disable_heuristics=4
[CPU Load Reduction]
play_nice_start=-1
play_nice_end=-1
play_nice_sleep=10
[Input Correction]
max_repair_size=200
ms_audio_delay=5
volume_slip=20
variable_bitrate=1
skip_b_frames=0
[Aspect Ratio]
ar_delta=0.08
cut_on_ar_change=1
[Global Removes]
padding=0
remove_before=0
remove_after=0
delete_show_after_last_commercial=0
delete_show_before_first_commercial=0
delete_show_before_or_after_current=0
delete_block_after_commercial=0
min_commercial_break_at_start_or_end=39
[USA Specific]
intelligent_brightness=0
black_percentile=0.0076
uniform_percentile=0.003
score_percentile=0.71
[Main Scoring]
length_strict_modifier=3
length_nonstrict_modifier=1.5
combined_length_strict_modifier=2
combined_length_nonstrict_modifier=1.25
ar_wrong_modifier=2
excessive_length_modifier=0.01
dark_block_modifier=0.3
min_schange_modifier=0.5
max_schange_modifier=2
logo_present_modifier=0.01
punish_no_logo=1
[Detailed Scoring]
punish=0
reward=0
punish_threshold=1.3
punish_modifier=2
reward_modifier=0.5
[Logo Finding]
border=10
give_up_logo_search=2000
logo_max_percentage_of_screen=0.1
ticker_tape=0
ignore_side=0
subtitles=0
logo_at_bottom=0
logo_threshold=0.75
logo_filter=0
aggressive_logo_rejection=0
edge_level_threshold=5
edge_radius=2
edge_weight=10
edge_step=1
num_logo_buffers=50
use_existing_logo_file=1
two_pass_logo=1
[Logo Interpretation]
connect_blocks_with_logo=1
logo_percentile=0.92
logo_fraction=0.4
shrink_logo=5
before_logo=0
after_logo=0
where_logo=0
min_black_frames_for_break=1
[Closed Captioning]
ccCheck=0
cc_commercial_type_modifier=4
cc_wrong_type_modifier=2
cc_correct_type_modifier=0.75
[Live TV]
live_tv=0
live_tv_retries=4
dvrms_live_tv_retries=300
standoff=0
dvrmsstandoff=120000
require_div5=0
div5_tolerance=-1
[Output Control]
output_default=0
output_plist_cutlist=0
output_zoomplayer_cutlist=0
output_zoomplayer_chapter=0
output_vcf=0
output_vdr=0
output_projectx=0
output_avisynth=0
output_videoredo=1
videoredo_offset=2
output_btv=0
output_edl=1
output_edlp=0
output_bsplayer=0
output_edlx=0
output_cuttermaran=0
output_mpeg2schnitt=0
output_womble=0
output_mls=0
output_mpgtx=0
output_dvrmstb=0
output_dvrcut=0
output_framearray=0
output_debugwindow=0
output_tuning=0
output_training=0
output_false=0
output_aspect=0
output_demux=0
output_data=0
output_srt=0
output_smi=0
output_timing=0
delete_logo_file=0
cutscene_frame=0
cutscene_dumpfile=""
cutscene_threshold=10
cutscenefile1=""
cutscenefile2=""
cutscenefile3=""
cutscenefile4=""
cutscenefile5=""
cutscenefile6=""
cutscenefile7=""
cutscenefile8=""
windowtitle="Comskip - %s"
cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25  /Z %2 %1"
avisynth_options="LoadPlugin(\"MPEG2Dec3.dll\") \nMPEG2Source(\"%s\")\n"
dvrcut_options="dvrcut \"%s.dvr-ms\" \"%s_clean.dvr-ms\" "
[Sage Workarounds]
sage_framenumber_bug=0
sage_minute_bug=0
enable_mencoder_pts=0

First PAT [   0] [ 100] 
[ 100] Added PMT[1001] pid = 1011 type =  27
[ 100] Added PMT[1001] pid = 1100 type = 129
       0:Auto selected video pid = 1011
       0: Auto selected audio pid = 1100

Gerry

[erik]

This is different as comskip did not finish process but may have crashed.

[gplasky]

Originally Posted by erik

This is different as comskip did not finish process but may have crashed.

OK. Thanks. Tonight is a new episode so I'll see if it works any better.

Gerry

[kemputer26]

Originally Posted by erik

The last two lines suggest Comskip is going into interactive UI mode.
Do you have the -w switch in the command line?
Do you run ComskipGUI.exe instead of Comskip.exe
Do you have
output_debugwindow=1
in your comskip.ini file?

-w in command line? No
Running ComskipGUI? No
output_debugwindow=1? Yes (I can't believe I didn't think of that...changed to 0, and now it's working perfectly).

Thanks!
Jeff