Does a network encoder actually encode?

[stryker]

I have a sagetv server and sagetv server setup as a network encoder. It all works fine however I notice that when my htpc sagetv client is viewing live tv fed from the card on the encoder, looking at both server's cpu usage it looks like the main sagetv server is doing the actual encoding of the stream and storing to the recording directory?

Am i right? is it not possible to make the network encoder actual encode the stream to spread the cpu usage?

[gplasky]

My understanding is using a server as a newtwork just allows it to claim the tuner as its own, so to speak. It makes the tuner installed in the box seem as if it is installed in the main SageTV server, makes it a part of the guide and controls everything it does. I would think if you had defined a recording directory on that box it would record it there but it would be driven by the main SageTV server. The encoding of the recording (for analog tuners) should be done on the tuner card because they are hardware encoders (or should be). Any HD digital streams come across as the final format and requires no encoding. The HD-PVR encodes in h.264 in hardware and again should not cause much of a load on the CPU.

Gerry

[stuckless]

A Sage network encoder is probably a misnomer for what it does. It's actually a "Network Tuner". The API is fairly generic, and it basically support changing channels and writing the vidoe output to a specified file location. There is nothing the API for actually performing an encode (although the tuner could and probably does encode as it writes the file).

Also, there is nothing in the API for transferring of video which means that either sage uses an undocumented way of moving the recorded file from server a to server b, or both server a and b share the same filesystem. (local or remote)

If sage is actually encoding the file again, on the local box, then it may be that a specific recording quality is set, and causing sage to re-encode the video again.

[stryker]

Hi,

Yes on my network encoder... tuner :-) I set the tvrecord directory to be the tvrecord folder on the main sagetv server. While watching a channel served by the network tuner I ran a file util on the main sagetv server and it showed the local sagetv process on the main sagetv server was actually writing the file, so it seems the network tuner streams the raw data from the tuner card to the sagetv server where it is encoded (or maybe thats transcoded) to the set recording quality and written to file where the sagetv client picks it up for display.

I can see this being a limitaiton if all the processing power has to be on the main sagetv server.... I saw one thread on these forums where a user was talking about encoding the stream from 60 tuners.... no number of network tuners in that instance would help.

I only use a network tuner as the haupp s2 card will not work in my main sagetv server so I'm happy, I was just curious.

Thanks for the replies,

Jon

[stanger89]

Originally Posted by stryker

Hi,

Yes on my network encoder... tuner :-) I set the tvrecord directory to be the tvrecord folder on the main sagetv server. While watching a channel served by the network tuner I ran a file util on the main sagetv server and it showed the local sagetv process on the main sagetv server was actually writing the file, so it seems the network tuner streams the raw data from the tuner card to the sagetv server where it is encoded (or maybe thats transcoded) to the set recording quality and written to file where the sagetv client picks it up for display.

You sure it was writing the file? I'm 99% sure the network encoder writes the file to the directory Sage specifies. For example SGraphRecorder uses normal Dshow filters and nothing special to record a file, it takes care of everything from tuning to writing.

At most, the network encoder streams the encoded stream back to the server to be written, but I don't think even that is the case. The server doesn't have any knowledge of the capabilities of the network tuners, beyond they exist.

Are you sure the server is writing the file? Not just reading it? The main server does monitor the recording file to make sure it's recording and restart the capture if necessary.

Also you don't mention which tuner(s) you have. Almost all of the supported tuners either have hardware encoders or are digital tuners which require no encoding.

I only use a network tuner as the haupp s2 card will not work in my main sagetv server so I'm happy, I was just curious.

You're talking DVB-S2 right? There's no encoding going on with that, it's just a pure bitstream capture.

[stryker]

Hi, thanks for your thoughts on this. I thought given you set the recording quality in sage then there is some encoding going on, in fact I would of thought in the recording quality option there should be one that states "use source".

I ran a file util that did show the locate sagetv process with the file open, however given what you've said I will check if its open for read or write.

However I can't dismiss the fact I noticed the main sagetv server was using around 20% cpu while the client was watching a channel from the net encoder. I'll do some tests and double check.

I have a haupp nova dual dvbt card in the main sagetv server and yes the haupp dvb-s2 in the network encoder/tuner??

Jon

[stanger89]

Originally Posted by stryker

Hi, thanks for your thoughts on this. I thought given you set the recording quality in sage then there is some encoding going on, in fact I would of thought in the recording quality option there should be one that states "use source".

Recording quality is sort of a fallback, it's ignored for digital tuners, and only used if Sage has to fall back to an encoder that supports bitrate selection.

I ran a file util that did show the locate sagetv process with the file open, however given what you've said I will check if its open for read or write.

Are you using SageTV as an encoding server, or a 3rd party network encoder? It's possible that Sage does stream the (encoded) data back to the main server and writes it there. But I know 3rd party network encoders don't work like that.

However I can't dismiss the fact I noticed the main sagetv server was using around 20% cpu while the client was watching a channel from the net encoder. I'll do some tests and double check.

For some reason, my server seems to run a "noticable" CPU usage most of the time.

[stuckless]

one intersing apect of the network encoder api is that there is a command "get file size" that is called from the sever to the the encoder server. basically the encoding server tells how large the file is getting..... In my experience, i've seen where the sage server gets overly agressive in calling "get file size" so much to the point that I was receiving about 30+ requests per second. In my network encoding server, I had to write some code to "block" and hold the connection so that I could "tame" the server. I basically put in code that said, if the time between this request and the previous is less than a specified # of seconds, then block and hold, and the reply.

I wonder if the cpu spike is related to this "get file size" command??

[stryker]

I ran the test, funny, I noticed cpu usage was only at around 5% while watching a SD channel that was served by the encoding server.

In answer to your question I have a sagetv server with a haupp nova dual tuner and a sagetv server acting as network encoder with a haupp dvb-s2 tuner (I have to do it like this as the s2 tuner will not work in the main sagetv server).

I ran the file util on the sagetv server while the client (a third pc with sagetv-client installed) was watching a channel served by the network encoder... it showed the main sagetv server doing writes to the file, so it is the main server writing the file, not the encoding server. As previously stated the network encoding server is using the main sagetv server's tvrecord folder over a network share.... although it looks like thats not neccessary.

So, stuckless, given my original question and what you've said; am I right in saying its the master sagetv server that writes the file for live tv?

[stuckless]

So, stuckless, given my original question and what you've said; am I right in saying its the master sagetv server that writes the file for live tv?

I could very be true... but if they are, then sage is using a proprietary undocumented means of transfering the file from server a to server b. The network encoder api does not provide the ability to do this, but that doesn't mean that Sage isn't doing it

[stanger89]

Well, we pretty much know the encoding server protocol is slightly different than the "normal" network encoder protocol, for example encoding servers are automatically discovered and enabled, while "network encoders" require manual file editing.

I'd not surprised if Sage uses a more sophisicated interface privately, one that would eliminate the troubles that can be associated with recording paths on different machines.

[stuckless]

...while "network encoders" require manual file editing...

Yes and no.... there is no out of the box api for discovery, but sage uses a udp discovery packet... i added it my code once and my sage server picked up and used my encoder without a manual configure.

[Nightrader_]

Originally Posted by stryker

I ran the file util on the sagetv server while the client (a third pc with sagetv-client installed) was watching a channel served by the network encoder... it showed the main sagetv server doing writes to the file, so it is the main server writing the file, not the encoding server. As previously stated the network encoding server is using the main sagetv server's tvrecord folder over a network share.... although it looks like thats not neccessary.

If your encoder is saving to a network share, then of course the server will be doing the writes to the file, as the client machine is only passing the file data to the SMB service (file sharing service), but this is not anything thing to do with SageTV. Anytime you write to a share, the server is the one actually writing to disk.

Cheers,

Gene Reeves

[stryker]

No, the fileutil showed the master server's sagetv process doing the writing.

[phelme]

hmm. I thought the only file size call was the 5 minute "are we still communicating?" check the master server does to the network encoding box to see if everything has gone to heck. that's what I thought I was seeing when trying to debug a problem I was having with the HD-PVR as a network encoder.

(a problem which neither i nor Sage support every did figure out BTW. I gave up and moved the DirecTV box to the server room. As far as I know, the HD-PVR, with current software, flat out won't function well as a network encoder.)

Originally Posted by stuckless

one intersing apect of the network encoder api is that there is a command "get file size" that is called from the sever to the the encoder server. basically the encoding server tells how large the file is getting..... In my experience, i've seen where the sage server gets overly agressive in calling "get file size" so much to the point that I was receiving about 30+ requests per second. In my network encoding server, I had to write some code to "block" and hold the connection so that I could "tame" the server. I basically put in code that said, if the time between this request and the previous is less than a specified # of seconds, then block and hold, and the reply.

I wonder if the cpu spike is related to this "get file size" command??

[nielm]

By the way, it is extremely unlikely that the network encode is passing raw video to the server because, firstly it would require a huge amount of sustained network bandwidth (raw SDTV = ~30MBytes/s, which is stretching the limits of a Gigabit LAN), and secondly, because your tuner card can not actually get raw video from an already-encoded DVB stream!
The 'recording quality' is also ignored for digital tuners for this reason.

What is the actual CPU usage that you see on the 3 computers?

[stryker]

Oh processor usage is fine on all three... they are all below/around 10%. I was only asking as I was curious, although this has turned into an interesting thread.

And yes of course all my streams are digital so now I know the "quality" setting is only used for analogue encoding I guess no machine is encoding anything...

although I stick by my guns in stating it seems the master server writes the actual file to disk, at least thats what filemon showed me.

I am planning on using a spare 100GB partition on my encoding server to add to the list of shares used for recording TV..... (formatted 64K of course)... I'll let you know whats writing what when I have a show received by my encoding server and also stored on my encoding server... I'd be a little disappointed if the show was streamed to the master server then written to the local share on the encoding server by the master server.... I'm sure that won't happen though :-)

[phelme]

Originally Posted by nielm

By the way, it is extremely unlikely that the network encode is passing raw video to the server because, firstly it would require a huge amount of sustained network bandwidth (raw SDTV = ~30MBytes/s, which is stretching the limits of a Gigabit LAN), and secondly, because your tuner card can not actually get raw video from an already-encoded DVB stream!
...?

I'm not familiar with the Hauppauge Nova Dual, but the HDHomeRun, in it's current incarnation, I think unleashes the largest stream at about 40 mbps. The SageTV server process has to spend some cycles on that converting it to an MPEG, but for my PC it's usually under 7% of the CPU (AMD 3600+ X2). When both tuners on the HDHR are going with HD material though, it averages around 9% with occasional spikes to 20%.

[stanger89]

It's not "converting" to MPEG, it already is MPEG from the HDHR. What the Sage server does (or maybe the HDHR, I'm not exactly sure about the data flow) is parse the MPEG Transport Stream, which contains multiple programs, and save the desired program as an MPEG Program Stream.

[phelme]

Originally Posted by stanger89

It's not "converting" to MPEG, it already is MPEG from the HDHR. What the Sage server does (or maybe the HDHR, I'm not exactly sure about the data flow) is parse the MPEG Transport Stream, which contains multiple programs, and save the desired program as an MPEG Program Stream.

that's what I meant... albeit lamely.