Odd problem with Network Encoder

[wnjj]

Originally Posted by panteragstk

Yup. Restarted sage. It still gives an error, but the recording doesn't die like it did. It will eventually continue.

I'll restart the encoding server and re-test and attach the logs.

Hmm. Maybe there's another bug behind the one just fixed. If turning fast switch off works you at least have a solution but if you're willing to keep debugging, so am I. The larger SageTV community thanks you.

[panteragstk]

Originally Posted by wnjj

Hmm. Maybe there's another bug behind the one just fixed. If turning fast switch off works you at least have a solution but if you're willing to keep debugging, so am I. The larger SageTV community thanks you.

Never stopped me before. Since I've used SageTV I've come across many a strange bug so this isn't new. It's just been a very long time since I've had any issues and I've gotten used to it just working without any interaction from me.

[Narflex]

KarlyFStein,

Are you sure you downloaded the latest code from GitHub before you built the AX file?

And if you did...can you build the Debug version of it and post that one instead? Then he can register that one, go to HKLM\Software\Microsoft\DirectShow\Debug\MPEG2Dump.ax (or something like that for the last part) and set the LogToFile value to be a log file. There may be other values to set there....I don't remember...it's been a long time...but then the filter can do its own logging and we will be able to find more specific details.

And of course I'd like to see the log files from using the newly built filter...because maybe there is also another problem and the issue with the DirectShow filter isn't a problem anymore.

[KarylFStein]

Originally Posted by Narflex

KarlyFStein,

Are you sure you downloaded the latest code from GitHub before you built the AX file?

And if you did...can you build the Debug version of it and post that one instead? Then he can register that one, go to HKLM\Software\Microsoft\DirectShow\Debug\MPEG2Dump.ax (or something like that for the last part) and set the LogToFile value to be a log file. There may be other values to set there....I don't remember...it's been a long time...but then the filter can do its own logging and we will be able to find more specific details.

And of course I'd like to see the log files from using the newly built filter...because maybe there is also another problem and the issue with the DirectShow filter isn't a problem anymore.

Yes, I did a sync and checked the source file before building:

//
// CloseFile
// Closes any dump file we have opened
//
HRESULT CMPEG2Dump::CloseFile( BOOL bAsync )
{
	CAutoLock lock(&m_fileLock);
	if (m_bRemoteFile)
	{
		CloseConnection();
	    m_hFile = INVALID_HANDLE_VALUE;
		return NOERROR;
	}
    if (m_hFile == INVALID_HANDLE_VALUE) {
        return NOERROR;
    }

	if ( !bAsync )
		CloseHandle(m_hFile);
	else
		AsyncCloseFile( );

    m_hFile = INVALID_HANDLE_VALUE;
    return NOERROR;

} // Open

The debug version is attached.

[wnjj]

Originally Posted by Narflex

KarlyFStein,

Are you sure you downloaded the latest code from GitHub before you built the AX file?

And if you did...can you build the Debug version of it and post that one instead? Then he can register that one, go to HKLM\Software\Microsoft\DirectShow\Debug\MPEG2Dump.ax (or something like that for the last part) and set the LogToFile value to be a log file. There may be other values to set there....I don't remember...it's been a long time...but then the filter can do its own logging and we will be able to find more specific details.

And of course I'd like to see the log files from using the newly built filter...because maybe there is also another problem and the issue with the DirectShow filter isn't a problem anymore.

It looks like maybe you can enable logging on the release version by creating a file called "MPEG2Dump_LOG.ENABLE" or does the registry key you mentioned still need to be there?

static void _flog_check()
{
	FILE* fp = fopen( "MPEG2Dump_LOG.ENABLE", "r" );
	if ( fp != NULL )
	{
		flog_enabled = true;
		fclose( fp );
	}
}

[panteragstk]

Originally Posted by KarylFStein

Yes, I did a sync and checked the source file before building:

//
// CloseFile
// Closes any dump file we have opened
//
HRESULT CMPEG2Dump::CloseFile( BOOL bAsync )
{
	CAutoLock lock(&m_fileLock);
	if (m_bRemoteFile)
	{
		CloseConnection();
	    m_hFile = INVALID_HANDLE_VALUE;
		return NOERROR;
	}
    if (m_hFile == INVALID_HANDLE_VALUE) {
        return NOERROR;
    }

	if ( !bAsync )
		CloseHandle(m_hFile);
	else
		AsyncCloseFile( );

    m_hFile = INVALID_HANDLE_VALUE;
    return NOERROR;

} // Open

The debug version is attached.

Would not let me register the debug version. Got an error. The previously attached version will register without issue.

[panteragstk]

See logs. Same issue.

[Narflex]

The MPEG2Dump_LOG.ENABLE file only enables a certain type of logging...but that may actually be useful, so it's worth a shot. There's more extensive logging that's done with the DirectShow debug facility.

And there definitely is a change in the behavior now. When it switches to the new file...it is reporting an updated, smaller file size. However, that small size doesn't grow at all for whatever reason. I'll take another look at the code again and see what I find (I didn't dig beyond that first issue before because it was so clearly causing a major problem)...but if you can get that other log file as well that could likely help out even more.

[panteragstk]

Originally Posted by Narflex

The MPEG2Dump_LOG.ENABLE file only enables a certain type of logging...but that may actually be useful, so it's worth a shot. There's more extensive logging that's done with the DirectShow debug facility.

And there definitely is a change in the behavior now. When it switches to the new file...it is reporting an updated, smaller file size. However, that small size doesn't grow at all for whatever reason. I'll take another look at the code again and see what I find (I didn't dig beyond that first issue before because it was so clearly causing a major problem)...but if you can get that other log file as well that could likely help out even more.

I'll do that no problem, but the debug version wouldn't register so I guess something went wrong with it.

[KarylFStein]

Originally Posted by panteragstk

I'll do that no problem, but the debug version wouldn't register so I guess something went wrong with it.

I'm afraid that I'm no VS expert. In fact I've only used it a couple times for SageTV . What I did is select Debug instead of Release in the pull down and rebuilt the MPEG file, (which spit out two files: the .ax and a .pdb). If there's something else that I should have done let me know.

[panteragstk]

Originally Posted by KarylFStein

I'm afraid that I'm no VS expert. In fact I've only used it a couple times for SageTV . What I did is select Debug instead of Release in the pull down and rebuilt the MPEG file, (which spit out two files: the .ax and a .pdb). If there's something else that I should have done let me know.

Me neither. No clue what it is supposed to spit out.

[wnjj]

Originally Posted by KarylFStein

I'm afraid that I'm no VS expert. In fact I've only used it a couple times for SageTV . What I did is select Debug instead of Release in the pull down and rebuilt the MPEG file, (which spit out two files: the .ax and a .pdb). If there's something else that I should have done let me know.

Normally that's all you need to do but I do see in the project file that the Debug and Release versions have different options set. You can find those in Project|Properties, then under Configuration Properties. For each section like C/C++ and Linker you can find an All Options section.

Likely one or more those those settings was changed in the Release version but not in Debug.

Then again, there may be a missing dependency like runtime Debug libraries that aren't included in the installer.

[wnjj]

Originally Posted by Narflex

The MPEG2Dump_LOG.ENABLE file only enables a certain type of logging...but that may actually be useful, so it's worth a shot. There's more extensive logging that's done with the DirectShow debug facility.

And there definitely is a change in the behavior now. When it switches to the new file...it is reporting an updated, smaller file size. However, that small size doesn't grow at all for whatever reason. I'll take another look at the code again and see what I find (I didn't dig beyond that first issue before because it was so clearly causing a major problem)...but if you can get that other log file as well that could likely help out even more.

I noticed the change in behavior too. It's starting to make a little more sense. When the last GET_FILE_SIZE call for the previous show comes in after the SWITCH, it returns 0 because it's no longer in the captureMap. Therefore it falls into the java.io.File.Length() 'else' statement in EncodingServer.java which returns 0 because the filename includes "stv://" and therefore not valid locally.

Then the next GET_FILE_SIZE with the new file returns the size of the old file because the switchover hasn't yet happened inside the 5 second m_switchNotify wait for the transition point in MPEG2Dump.cpp.

Then, about 1 second later GET_FILE_SIZE for the new file returns a much smaller, non-zero number which I suspect is the size of the write after the transition that outputs the remaining buffer contents in MPEG2Dump.cpp:

return m_pDump->Write( pbData + tranny, currLen - tranny );

The fact that the old file size is no longer returned means the OpenFile function is now completing (which it didn't with the remotefile=false bug) and setting m_shareInfo.totalWrite back to 0. The only places that subsequently increment that file size variable are in the Write() and Seek() methods, one of which I believe is being called above right after the switchover OpenFile.

After the switch, subsequent writes come through the Receive() function eventually ending up at the bottom at this statement:

return m_pDump->Write( pbData, currLen );

It's obviously not getting there or the file size should be growing.

Is it somehow stuck repeatedly calling Seek() at setting the point/filesize to the same place? Actually, It doesn't look like Seek() is ever used in this mode.

[wnjj]

Hey KarylFStein,

If you want to compile another option, I've attached a modified MPEG2Dump.cpp where I turned all of the dbglog (resistry-based logging) calls into the local ones and forced it on. This way everything will log into a local "MPEG2Dump.LOG" file.

You can compile it in Release mode and it should just drop in again. Note: I have not compiled it so hopefully didn't introduce any syntax errors.


Panteragstk,

This should not be used log term since it may produce a ton of data and there's no limit on the log file. Just run it long enough to capture another one of these failures. It should operate just like Jeff's fixed one, but with logging. Back up the previous one first.

[panteragstk]

Originally Posted by wnjj

Hey KarylFStein,

If you want to compile another option, I've attached a modified MPEG2Dump.cpp where I turned all of the dbglog (resistry-based logging) calls into the local ones and forced it on. This way everything will log into a local "MPEG2Dump.LOG" file.

You can compile it in Release mode and it should just drop in again. Note: I have not compiled it so hopefully didn't introduce any syntax errors.


Panteragstk,

This should not be used log term since it may produce a ton of data and there's no limit on the log file. Just run it long enough to capture another one of these failures. It should operate just like Jeff's fixed one, but with logging. Back up the previous one first.

I will do that once it is compiled.

[KarylFStein]

Originally Posted by wnjj

Hey KarylFStein,

If you want to compile another option, I've attached a modified MPEG2Dump.cpp where I turned all of the dbglog (resistry-based logging) calls into the local ones and forced it on. This way everything will log into a local "MPEG2Dump.LOG" file.

You can compile it in Release mode and it should just drop in again. Note: I have not compiled it so hopefully didn't introduce any syntax errors.

Well, it sounds like you know what you're doing, have the environment and should be doing the compiles , but here you go.

[wnjj]

Originally Posted by KarylFStein

Well, it sounds like you know what you're doing, have the environment and should be doing the compiles , but here you go.

I have a compiler but haven't compiled the project before. Plus I'm in an airport right now.

Thanks for the help.

[panteragstk]

Got the mpeg dump log. Let me know if anything else is needed.

[Narflex]

OK, I think I found the next problem. Hopefully this should do it. Before I go committing this...can you build this one Karly and then post the new .ax file for panteragtsk to try?

Thanks for adding that extra logging...it gave me the info I needed to figure out the problem.

The new issue was that it was using the wrong filename for the remote upload. It was setting the nextFilename BEFORE it was stripping out the stv:// stuff from the URL when it was doing a switch so it was sending an stv:// URL to the MediaServer to open rather than a normal file path.

[KarylFStein]

Originally Posted by Narflex

OK, I think I found the next problem. Hopefully this should do it. Before I go committing this...can you build this one Karly and then post the new .ax file for panteragtsk to try?

Thanks for adding that extra logging...it gave me the info I needed to figure out the problem.

The new issue was that it was using the wrong filename for the remote upload. It was setting the nextFilename BEFORE it was stripping out the stv:// stuff from the URL when it was doing a switch so it was sending an stv:// URL to the MediaServer to open rather than a normal file path.

Sure, attached.

One small change on line 846:

- if (m_pFilename) {
+ if (m_pFileName) {