STB Firewire (IEEE-1394) Support

[foolio]

Dan:

i also get a ton of those "get_file_size" messages. However, I can view live HD without any problems. I watch the tonight show live each night. This is over a network - with the firewire source in one computer, the server in another computer, and I'm watching it on a client. No problem, but tons of "get_file_size'

i'm using the nvidia decoder with overlay

[DFA]

Hi everyone.

Just arrived in Cairo and have a connection here at the hotel. So I get one last chance to feed my Forum Jones before moving out to the boonies. Then it will be Forum withdrawels.

Spaceghost:
Are you using seperate emitter or the USB-UIRT main blaster? Try different orientations and consider possible reflections from things around the room. The main blaster is fairly strong and too close might cause difficulty. Before I got seperate emitter, I used to put the USB-UIRT sideways to the SA STB.

When learning codes to USB-UIRT, as noted by dvsing, 3 to 4 inches is about right and normal behavior. IIRC, USB-UIRT uses a different and dedicated receiver for learning that is not very sensitive by design. Repeat values of 10 are normal to get learning bar to go all the way across (or is it up?. If the learning bar goes all the way, no further increase in repeat values are needed or will do any good. For remotes that do not repeat, there is a provision to accept the single burst.

Anyway, might try working with orientation.

DFA

[dvsing]

Originally Posted by foolio

Dan:

i also get a ton of those "get_file_size" messages. However, I can view live HD without any problems. I watch the tonight show live each night. This is over a network - with the firewire source in one computer, the server in another computer, and I'm watching it on a client. No problem, but tons of "get_file_size'

Aha!.... it took me some further debugging but I think I know why in your situation the rapid-fire GET_FILE_SIZE messages are not a problem, and in my situation they are.

After some more inspection, I noticed that each one of these GET_FILE_SIZE messages is being issued on a separate connection. This is because the SGraphRecorder disconnects the TCP socket after reception of a command (I saw this in the source code).. I don't know the SageTV remote encoder protocol well enough to know if this is mandatory (to disconnect) or optional. In any case... what's happening is that every time these TCP sockets are closing, they go into the TIME_WAIT state (as they should) and for a few minutes they prevent the OS from re-using that outbound port. So, in my situation, since I'm running SGraphRecorder and SageTV on the same system, SageTV is able to make this transaction with SGraphRecorder many times per second.. 30-40... What is happening is that it's going so fast that it's putting all of the available outbound ports into TIME_WAIT and then no connections can be made until these exit the TIME_WAIT state.

In your situation, it's working OK because SGraphrecorder is on a separate system from SageTV, and the small amount of network latency reduces the number of times per second that SageTV can make these GET_FILE_SIZE requests, and it never gets a chance to put ALL of the outbound ports in the TIME_WAIT state because by the time it wraps around the ports have had time to exit the TIME_WAIT state.

If all of this TCP tech stuff doesn't make sense, it's sufficient to say that the problem is caused by a few contributing factors:
1) SageTV seems to send these GET_FILE_SIZE messages as fast as possible-- this is probably because the playback function just makes a call to a generic get_file_size() function which normally just checks the file size locally, however with network encoders it has to go thorugh the process of sending a TCP message.. much more overhead-- I suggest that if working with a network encoder these GET_FILE_SIZE messages should be limited to perhaps once a second....
2) Normal behavior of the TCP stack, which puts ports into TIME_WAIT after they are closed for a few minutes to avoid confusing stale packets with new connections.
3) SGraphRecorder closes the TCP socket after reception of a command-- I dont know if this is mandatory or optional. If SGraphRecorder and Sage were able to re-use the same TCP connection for all communication, then this high rate of GET_FILE_SIZE messages would not eventually stall the TCP stack because there would be no continual creation and destruction of TCP sockets.

-Dan

[foolio]

Dan:

your observations seem correct. When I run a sage-client on the same system that the UNE is on, it does stutter and slow down real bad.. if I'm watching on a different client, all is well.

[DFA]

dvsing:

You may indeed be on to something and understand what you are saying and looks like an issue which needs addressing. However, I am using the UNE in local connection and at least one other person (travisbell) and for some reason or another have managed to escape this. I have had it in use for close to a month including a period of pre-release testing and we use it successfully for both "Live TV" viewing and recording.

No doubt the UNE is subject to further refinement and enhancement and problems not previously encountered yet to surface. Anders makes the source available for those who wish to advance it. I'm sure Anders himself will update the UNE when advised of problems or needed enhancements.

It might be a good idea to let him know of your findings and thoughts. Anders is very familiar with how Sage interfaces and uses network encoders since first working with the DVB network encoder and evolving with changes to the Sage network encoding specs. He would also be a good source of information so that you could directly implement changes you deem needed.

I look forward to any advancements and improvements to the UNE!

EDIT: Interestingly, I have not seen anything near this high a polling frequency (Live TV or otherwise) and would certainly explain why not encountering the problem myself. Your analysis and conclusions make sense based on this rapid a polling rate. This would seem to indicate there may possibly be a Sage properties configuration parameter involved. I am more curious as to the source of why such differences in Sage's polling behavior exist. I do not understand the reason that Sage would need this kind of frequency of file size update to begin with. I could see this being a problem for Frey's own "SageRecorder" network encoder. Jeff would be best source of advise on that subject. If the polling rate can be reduced to a managable level, problem solved. Also, be aware that changes occurred to Sage's network encoder protocol somewhere in the developement of 2.0.x. I have used the UNE with 2.2.4 and 2.2.5 RC.

DFA

[dvsing]

DFA -

I've asked around and you're right--- this high frequency of GET_FILE_SIZE requests doesn't seem to be the behavior of every system.. and I'm curious why. On the thought that it was related to a legacy *.properties setting, I backed up and erased both of my .properties files (sage/sageclient) and allowed the latest Sage beta to re-write them from scratch. After spending time getting my encoders reconfigured, etc, the same high-frequency polling still existed....

I have emailed Frey Tech. about this... not necesarily a bug report, but digging for information as to what is driving these GET_FILE_SIZE requests and why there are so many-- I don't think they would need to occur any more than once or twice a second really....

Another odd experience I had this evening with the UNE was that I had set it up to record a program from 10pm to 11pm. In the UNE log, it shows that it was instructed from sage to start at 10pm-- all is good. Then, for some VERY ODD reason, there is a command from sage to stop the recording at 10:10pm, and then it restarted the recording immediately following, (using a -1 part extension on the file)... When I go to play it back, sage doesn't even see the first 10 minutes of the program anymore, just the last 50 (the second part after the stop/start). I'm not sure why it did this... I have to look into it further when I match up the sagetv debugging logs to the UNE debug trace.

-Dan

[spaceghost]

I'm using a separate blaster with the UIRT. I even tried to hide the main UIRT as well as set it in front of the STB as well. I don't understand what's going on with my number 6...I'll keep learning it until it gets happy. I see the lights blink on both blasters but the STB is not accepting the signal. I understand what the repeat does now, thanks DFA.
The GET_FILE_SIZE wouldn't have anything to do with firewalls would it? I wouldn't be surprised if windows' firewall had something to do with it. Some of these damn shareware games my kids download hits zone alarm like crazy when I have it blocked from trying to get out to the internet. Would be nice if it were that simple.

[dvsing]

Spaceghost--

If you're having trouble learning remote commands into the UIRT, make sure that the remote has fresh batteries, and also try placing the remote/UIRT at different distances and angles with respect to each other. There was one command (Power - didn't even need it, but was just testing things out) that I had trouble getting the UIRT to learn correctly.

-Dan

[AndersNolberger]

I don't now why Sage is sending so many File_size commands, my system does not. But I might be able to do something about the network connection being closed.

As you might have seen in the log, Sage is sending a version command to SGR right after start. That is a question to Sage Recorder about what version it is. If SGR answer 2.1 (I think) it will not close the connection. But the command is fairly new and I have not implemented it so Sage thinks it is an older SageRecorder that needs a new connection each time. I will look into answer to the version command and see if the behaviour is changed.

[spaceghost]

I've tried to relearn the code several times. Both moving in distance toward/away from the UIRT, angle, and even upside down Looking at the code, it's the same every time. If this doesn't work, I'll try to pull the code from my Pronto.

So I was thinking, my HD channels are all in the 6xx range. What if the 6 isn't messed up; what if it's something with the first digit it tries to send? So I created a multigroup and assigned that to Number6. The first code it sends is enter...which does nothing on the STB if you press it by itself. The second code is the 6. I'll keep playing with it, but it's looking positive. I see it trying to send the enter but the STB doesn't recognize it. Then it sends the 6 and it sees it so far. But as always, things change and I might be back to the drawing board. I need this to work 100% of the time for it to be useful.

FWIW, I'm running sage 2.2.5, girder 3.3.0, sgraphrecord 1.06...and I always see the Number6 command come thru on sgraphrecord.

I was trying to record LOTR last night on the STB and thru SAGE. Guess that didn't work out so well. Sage was trying to turn the channel but it couldn't...kept sending a Number2 command over and over every 5 minutes...so I recorded 3 hrs of the weather channel. Don't know why it kept changing to channel 2...it was supposed to be recording on channel 612. I did notice it tried to change to channel 612 at the beginning..but from then on out..channel 2. Guess I'll really need to keep the STB from recording at the same time as Sage while I'm still in the testing phase.

[Stealth1971]

I brought this up in another thread @
http://forums.sage.tv/forums/showthr...7&page=3&pp=20

I have created an app that has programmatic control over ATI HD Wonder BDA filters.. (builds the graph, changes channels, etc) and I have 80% of Network Encoder server completed. (I have been so busy this has taken a back seat )

I am having some issues with playback - check the about link to for more info... Very close to getting this done - any feedback will probably get it completed...

Thanks

[dvsing]

Originally Posted by spaceghost

I was trying to record LOTR last night on the STB and thru SAGE. Guess that didn't work out so well. Sage was trying to turn the channel but it couldn't...kept sending a Number2 command over and over every 5 minutes...so I recorded 3 hrs of the weather channel. Don't know why it kept changing to channel 2...it was supposed to be recording on channel 612. I did notice it tried to change to channel 612 at the beginning..but from then on out..channel 2. Guess I'll really need to keep the STB from recording at the same time as Sage while I'm still in the testing phase.

Aha!!

This is a bug in Sage I'm pretty sure-- I see the exact same problem... sometimes Sage wants to tune the network encoder to channel 2 instead of the the proper channel. If you look in the sage debug log, you can see that indeed it is telling the UNE to tune to channel 2 which is incorrect (in my situation it should have tuned 709. Not a 2 in that at all!)

I am reporting this bug to Frey, along with the sage logfile showing the mistake.

Often when trying to figure out where the problem is, it helps to look at not just the UNE log, but the sage log(s) as well...

-Dan

[dvsing]

Originally Posted by AndersNolberger

I don't now why Sage is sending so many File_size commands, my system does not. But I might be able to do something about the network connection being closed.

As you might have seen in the log, Sage is sending a version command to SGR right after start. That is a question to Sage Recorder about what version it is. If SGR answer 2.1 (I think) it will not close the connection. But the command is fairly new and I have not implemented it so Sage thinks it is an older SageRecorder that needs a new connection each time. I will look into answer to the version command and see if the behaviour is changed.

Anders--

It looks like in your network code you call "Disconnect" after reception of a command. It will probably take a combination of version reporting and perhaps a small code change in your tcp server code.

This would be *great* if Sage were able to use the same socket to communicate with UNE.

On another note, I noticed last night that I had a program set to record from 10pm-11pm from the UNE. At 10pm, the UNE was told to begin recording, which it did. at 10:05pm, the file size was observed by sage to be 26k.... after 5 minutes, this doesn't make sense. At 10:10pm, the file size was again observed by sage to be 26k-- since this was the same size as at 10:05, there is a note in the sage log that it assumed the remote encoder had halted, for some reason... so it sent a STOP command and restarted the encoding-- from that point on, it worked fine, but I lost the first 10 minutes of the program.

Now, since the file size only ever got to 26k, I assume that the recorder was only active for a fraction of a second... is this perhaps related to a channel change? I have zero delay set for tuning delay... Although I could probably correct for this problem with tuning delay, I have a suggestion for UNE:

When a recording is started, check the file size ... maybe once per second for 5 seconds, and make sure that the size is always increasing.... If not, assume the graph froze for some reason and restart the graph-- this way we wouldn't have to wait for two 5-minute sage polling cycles for sage to detect this problem and restart the graph.

This is more of a CYA feature for UNE... Tuningdelay is probably the proper way to fix this?

-Dan

[jvheitz]

dvsing,
I am also running UNE on the same system as sage without any problems. I do not notice the numerous GET_FILE_SIZE requests that you are seeing.
So far, I have not had many problems (other that the Audio error previously noted and corrected)
Hopefully we can just figure out a way to change the channel over firewire.

[dvsing]

Update on my experiences thus far:

* Still waiting for a response from Frey regarding GET_FILE_SIZE command rate to network encoder during live playback.

* I found that increasing the priority of the SGraphRecorder process resulted in video that played back with practically no stutter at all. -- now this is something that I don't quite understand. Unless SGraphRecorder is experiencing buffer overflow or dropouts, there shouldn't be a problem with the recording, right? My experience seems to dictate otherwise.. I was experiencing stuttery playback, and I noticed that stuttering video is most obvious in scenes that are very slow pans (like a pan over a landscape or something). When I played back the video multiple times, stuttering occured in the exact same places... so this indicated to me that it might be a capture problem and not a playback problem. Sure enough, increasing sgraphrecorder priority seemed to create much better results on playback...

Again, this doesn't make sense to me-- I thought SGraphRecorder just used the graph we have now, which takes the TS stream, demuxes it, muxes it as PS and dumps it to a file... As long as the buffers are adequate then momentary CPU spikes shouldn't damage the recording... but apparantly it does.. strange.... also, how are the rest of you clocking your graphs?

I'm using this for my graph:
Mot6200 -> NVIDIA TS Info Parser -> MPEG DeMux -> Cyberlink MPEG Mux -> Cyberlink MPEG Dump.

I've read that for directshow capture graphs, you usually want the clock source to be coming from the capture source, and that if you don't override the clock this is the default behavior. I have the option to override the clock to be at the MPEG Dump filter or the MPEG DeMux filter. Various locations of the clock resulted in different behavior.. If I didn't override the clock at all then sometimes the graph just stalls. I ended up clocking the graph at the MPEG DeMux-- there should be a "right" way to do this... any DirectShow experts out there?

-Dan

[spaceghost]

Dan-
My computer is doing it now. No changes except for trying to plug a USB hub in between the UIRT. Took the hub out and it's still doing it. I get like 30 messages a second and it seems to be making the video stutter so horribly. I'll have to look thru the logs. The only thing I've done with the HD side of it was a failed recording today. Kept changing to channel 2 again.

[dvsing]

Spaceghost,

What do you mean by "doing it now" ? Do you mean you're seeing the out-of-control GET_FILE_SIZE messages to SGraphRecorder during live-tv watching?

-Dan

[spaceghost]

yes, GET_FILE_SIZE.

I tried to restart several times. Tried to remove all the channels. Tried to remove the tuner. rebooted several times in between. Then the UIRT wouldn't work. Man got me confused. So I added the tuner back and tried to set up the channels. That was ok except sage wouldn't show the new channels. So I rebooted one last time. Checked it when it came back up and it seems to be working normal now. Now I'm back to having it work..we'll see how long it lasts.

FYI, was getting exactly 27 GET_FILE_SIZE messages per second.

[dvsing]

I didn't bother to ever count them, but that sounds about right. Hmm... I hope that Frey gets back to me soon on what determines the rate of GET_FILE_SIZE messages sent to remote encoders when viewing the same file that they are recording. I sent a bug report/question to betatest@freytechnologies.com ... no response yet, I assume they are busy with other stuff =)

-Dan

[spaceghost]

my UIRT quit working, so I'm out of commission until Jon sends me a new one.