Utility: Comskip

[erik]

When comskip hits the end of the recording while the recording is still ongoing it will suspend itself and try again later. This is always enabled.
When live_tv=1 is set Comskip will always send out the commercials while they are found but with less accuracy.
When comskip has finished a complete recording it will always recheck quickly the total recording with full accuracy.
So the fastest and most accurate way is to run comskip is while the recording is ongoing. Comskip will finish the total recording a couple of seconds after the recording is finished.
As a bonus you can enable live_tv=1 allowing you to jump commercials in timeshift mode
If you want to see how this all is done you should enable live_tv=1 and run comskip with the debug window (-w switch)
COmmercials blocks will appear on the timeline when found and at the end a improved detection may change what is found.
A word of warning. In strictly regulated USA conditions the detection of commercials is so simple that there may be no difference between the live_tv detection and the total recording results.

[erik]

Do you guys use UNC or mapped drives?
Or is comskip always on the c: drive?
This because I did some test and in my setup without UNC comskip is able to find the ini next to the exe even when executes in another directory on a mapped drive so I am lost on how to solve this.

[blade]

Originally Posted by erik

COmmercials blocks will appear on the timeline when found and at the end a improved detection may change what is found.

Cool, that sounds perfect.

I don't watch many shows while they're still recording, but it's nice to be able to without sacrificing accuracy on the ones that I watch later.

[lovingHDTV]

Originally Posted by erik

Do you guys use UNC or mapped drives?
Or is comskip always on the c: drive?
This because I did some test and in my setup without UNC comskip is able to find the ini next to the exe even when executes in another directory on a mapped drive so I am lost on how to solve this.

I tried both ways with similar results. comskip is installed on the c: drive under my documents. While my vidioes are on a local F: drive and a UNC drive \\tower\disk4\SageTV.

[jlindborg]

Pretty sure the INI issue has nothing to do with Comskip. When run manually or with DirMon (classic) it works find finding the INI file in the same directory comskip.exe is in - DirMon2, however, causes problems. I'm guessing because the current directory is not being set properly when launching an app.

[ldavis]

Erik,

I have another problem that, although I think it is with the Sage plugin, you can resolve with an addition to comskip. The best way to describe it is to give an example. I am running comskip on “Numb3rs”. After completion, I dropped the .txt file back into comskip.exe to see the cut point. Well, they weren’t close – they were prefect with not even a frame of a commercial at any point. When, I played this back in Sage, the first commercials are OK but it got progressively worse later in the show.

Here is what I think the problem is. No recorded show has exactly the number of frames that the length would indicate. The “numb3rs” file had 108303 frames, which would indicate a length of 60 minutes and almost 14 seconds. I know that the Sage timeline shows this as exactly 1 hour but I do not know how it takes up the slack. I think that the Sage plugin simply divides the frames in the .txt file by 29.97 to arrive at the (in)correct point on the timeline. Some of the time when the actual length of the file is close to what it should be, this works fine. In others, such as my example above, there are errors. In most cases the errors are just enough to irritate the end user. Also these errors get progressively worse later in the file.

Here is what I propose ( And I have verified this works with hand calculations which I input into the”numb3rs” .txt file. In the .ini file have a flag set to 1 to perform the following adjustment to the final frame number cut points prior to writing out the file

F’ = F(T/Ta)

Where:

F’ = Corrected frame number
F = Uncorrected frame number
Ta = Actual time of the file in minutes = Total Frames/(60*fps)
T = Ta rounded up or down to the nearest minute

Please note that this calculation would need to be performed prior to the adjustment for 60fps files. If it is done after then the fps in the Ta equation would need to be 29.97 in the case of 60 fps files.

[erik]

Maybe its better if the Sage plugin gets corrected? Any chance for that?
This would already be the second bug in sage I have to correct for.

[ldavis]

I agree but you just happen to be more responsive. I also think that it would be difficult for Neilm to correct for this since I do not think that he knows total frames (or actaul length) when the calculation is done. I'll ask.

[lovingHDTV]

I remember at one point that you could tell comskip to mark the commercial with some padding. So before starting a commercial break you could mark the commercial 1 second after the "detected" mark, and 1 second before the end of the commercial break. This would allow you to see a commercial start and stop prior to the show changing. Is this still available?

Maybe it was a kludge was to address the issue that ldavis is talking of. I do see that my commercial marks tend to miss a few seconds of the show as they go forward through the show.

[erik]

Originally Posted by lovingHDTV

I remember at one point that you could tell comskip to mark the commercial with some padding. So before starting a commercial break you could mark the commercial 1 second after the "detected" mark, and 1 second before the end of the commercial break. This would allow you to see a commercial start and stop prior to the show changing. Is this still available?

Maybe it was a kludge was to address the issue that ldavis is talking of. I do see that my commercial marks tend to miss a few seconds of the show as they go forward through the show.

Did you read the tuning guide?

[erik]

Originally Posted by ldavis

I agree but you just happen to be more responsive. I also think that it would be difficult for Neilm to correct for this since I do not think that he knows total frames (or actaul length) when the calculation is done. I'll ask.

But then he should calculate in seconds and not in minutes

[erik]

Originally Posted by ldavis

I agree but you just happen to be more responsive. I also think that it would be difficult for Neilm to correct for this since I do not think that he knows total frames (or actaul length) when the calculation is done. I'll ask.

It is possible that you have a DTV recording with some bad MPEG spots (marked by "PTS jumped xx frames" in the log file)
Comskip now uses the PTS as a reference instead of the actual successfull decoded frames.
Some other players use the length of the file to position.
For DTV recordings with missing frames this will go progressively wrong.
Just a thought.

[ldavis]

Originally Posted by erik

It is possible that you have a DTV recording with some bad MPEG spots (marked by "PTS jumped xx frames" in the log file)
Comskip now uses the PTS as a reference instead of the actual successfull decoded frames.
Some other players use the length of the file to position.
For DTV recordings with missing frames this will go progressively wrong.
Just a thought.

That sounds like a good possibility. Problem is that with HD recording, you are going to have some bad frames so hopefully the STVi can be fixed to account for it.

[erik]

Or the stv can use the time format comskip can supply.
An example is the EDL format also used by mencoder and mplayer.

[erik]

From the changelog


Changes in 0.79.37
- Added parameter: drvcut_options to set the way the dvrcut .bat file is being generated. The default allows for direct execution without any parameters
- Bug solved: No audio decoded with AC3 in MPEG PS files.
- Bug solved: AC3 audio decoding unreliable
Changes in 0.79.36
- Added ini parameter, setting output_dvrcut=1 will enable the generation of the bat file that can run the commandline dvr-ms cutter to generate a clean dvr-ms file. You need dvrcut.zip for this
Changes in 0.79.35
- Bug solved: Commercial frame numbers found on dvr-ms files with logo where completely wrong.
- Bug solved: There is a difference in results when processing an mpeg or the corresponding .csv

[kha]

Originally Posted by erik

It is possible that you have a DTV recording with some bad MPEG spots (marked by "PTS jumped xx frames" in the log file)
Comskip now uses the PTS as a reference instead of the actual successfull decoded frames.
Some other players use the length of the file to position.
For DTV recordings with missing frames this will go progressively wrong.
Just a thought.

This is exactly the situation I have - lots of missing frames due to poor signal quality. Making STVi aware of non-perfect mpeg files would be the right solution.

Eric,

In the meantime since you have already introduced a correction in comskip for one bug in SageTV, can you make an additional correction based on ldavis's formula? This would be a great workaround!

MK.

[erik]

Its is possible to disable the timeline repair feature of Comskip by setting
max_repair_size=0
in the comskip.ini file
Could you try this?

[erik]

Build 38 is released
From the changelog

Changes in 0.79.38
- Added parameter: Setting sage_minute_bug=1 in the ini file will correct the frame numbers in the .txt file in a sage specific way.

Could you guys test it?
I can't and I even don't want

Be aware that when this is enabled it is NOT possible to edit the resulting frame numbers with the comskip cutpoint editor becuase you will get an accumulative correction, I am too layzy to implement the inverse correction.

[kha]

Erik,

I checked option max_repair_size=0 in the comskip.ini file. With this option neither Sage determines commercial position properly, nor can I see them identified properly in comskip itself.

I also tested the latest built of comskip with sage_minute_bug=1 and unfortunately Sage still does not determine commercials properly either...

On the second thought, are you or ldavis sure that Sage's comskip STVi uses rounded minutes for frame position calculation? I doubt it since this would result in commercial offset for practically all recordings. I see problem only with the recordings that have lots of missing frames. Other recordings are fine. Do you know how Sage seeks through the file? I am afraid the correction may be different throughout the file. I am guessing that at a given point of time it may have to be made based on run_time*fps vs. # of recorded frames. But this needs to be verified with the authors of Sage/Sage comskip STVi.

MK

[ldavis]

Erik,

I will check on the new parameter you have added as soon as I can, though I am not sure that I do not agree with you that the best approach is to try to get the authors to modify their STVi. I will need to run some comparitive cases with the flag on and off. I will let you know

Kha,

As far as exactly how the comskip STVi works, I am not completely sure of anything but here is what happens in two cases which Erik has helped with:

1.) 720p/60fps - Comskip correctly outputs the frames but since a frame rate of 59.94 is not an option with the STVi (only 29.97 and 25) then, using 29.97 the skip points on the timeline come out wayyyyy off. If a commercial was supposed to be from 5 minutes to 10 minutes then it appears on the timeline as from 10 minutes to 20 minutes. The STVi does not really contain an error in this case, but it is not flexible enough to handle multipe frame rates at the same time. With Erik's adjustment everything is right. That lead me to the conclusion that the STVi simply takes the frame numbers and divides by the frame rate (as selected by the user - not read from the mpg file) and uses this time directly as a cut point. There may be other possibile explanations but that is the only conclusion I could come to.

2.) I have been seeing a discrepancy with what the comskip debug window shows as the cut points and what the STVi shows. Normally comskip is dead on and the STVi is different, sometimes a lot and sometimes not so much. Looking at the .txt files, the number of frames is never exactly the length of the show. Now if I am correct about how the STVi calculates the cut times then these extra frames (presumably due to errors in the digital file) will create errors. Obviously not as bad and the 60fps issue but will make the commercial cuts wrong. I tested my theory with a hand modified file (using the equations I posted previously) and it produced perfect results - the comskip debug window and the STVi agreed completely. Now this error is more noticable with HD material since it probably has digital "transmission" errors that analog obviously does not have. I will bet that the with mpg files generated from analog tv, the overage/underage of frames is very minor and is not noticable. And maybe, my setup introduces more errors in the HDTV files than most. It may even be dependent on your local broadcaster

But, in answer to your original assertion that there is an offset to all comskip files. I think that the answer is all do have some error. Until we started seeing the large errors in the HD files, the errors were so minor that we never noticed.