New DVB(T/S/C) Network Recorder

[StephaneM]

Hi All,

As promised, I'm releasing today the first version (well it's a pre-release really) of my new network encoder for SageTV : LM DVB Smart Recorder.

Here is a summary of its features:

• DVB-T / DVB-S support (DVB-C wil be added later)
• CAM support (currently TechnoTrend (tested) and Digital everywhere, more hardware can be added, for instance WinTV USB CI)
• MDAPI support (untested, and you are basically on your own with this)
• Diseqc support (currently TechnoTrend (tested) and Digital everywhere, more hardware can be added later, motorized dish are not something planned in the near future)
• Subtitles extraction in real time for TTX and DVB Subtitles (same feature as DVBViewer Recorder)
• Smart Buffer engine : the smart buffer engine allows SageTV to not loose any content during back to back recordings.
• Very efficient disk recording (asynchronous I/O, write buffer that you can configure)
• Easy to use interface for managing recorders and adding channels
• LCN support (Logical Channel Number) : some DVB streams does have a LCN in the DVB Tables, but this is not really a standard (it works for me with DVB-T in france)
• Easy channel scan (usually the scan engine is smart enough to reconcile channels if a channel frequency changes or something like that: the channel list is color coded : green = new channel, red = missing channel, yellow = update channel)
• Sortable channel list
• Record only some channel in MPEG2 / MPEG4 (this is something very useful in France for some channels that sometimes airs in MPEG2 and sometimes in MPEG4 => this is something SageTV doesn't like at all)

Features to come in the near future:

• XMLTV Importer update for easy integration of lineups created by LM DVB Smart Recorder
• Online database for sharing channel lineups
• Update scan : for updating all your channels all at once (that is scanning all already used frequencies and trying to update the channels you choose to receive)

You can download LM DVB Smart Recorder or discuss it in its own forums here.

Note: LM DVB Smart Recorder share many things with DVBViewer Recorder for it configuration / subtitles support. As it is a pre-release my web site doesn't have many documentation yet about LM DVB Smart Recorder, but you can review some document about DVBViewer Recorder instead (for subtitles and SageTV setup)

Regards,
Stéphane.

[Whitey]

Great Program Stephane!

I'm going to try setting this up with Sky UK. I'm a long time user of DVBE4sage but thought I would try this.

One question, since there are no CAMs available for Sky UK, we have to use a MDAPI plugin called NDSCam with a serial card reader. How do I set this up with your software?

Thanks

Whitey

[oaamaas]

Tried the SmartRecorder on a WHS installation, SageTV 7.0.10 and dual Digital Everywhere Floppy DVB-T with CAM.

• Installation OK
• Configuration OK (five frequencies added, one at a time)
• Finally tried to add recorder to SageTV - fails

First time I tried to add the recorder, SageTV was running resulting in SmartRecorder error message "Stop SageTV before adding a recorder".
I then stopped it from the console and got something like "Failed to add recorder" (only saw this once) when I retried. After this I've tried to adjust the recorder config, and also tried several times to start and stop the service. SmartRecorder always responds with "Stop SageTV before adding a recorder", the state of the service does not influence this.

Hope this is something you can fix as DVB subtitles is the only thing I really miss - my WHS solution is otherwise fully functional and rock stable. Thanks!

[StephaneM]

Originally Posted by Whitey

One question, since there are no CAMs available for Sky UK, we have to use a MDAPI plugin called NDSCam with a serial card reader. How do I set this up with your software?

You need Agrawal's Directshow MDAPI filter (the same one used in MediaPortal / GBPVR) and set it up in the same way (on MediaPortal you can create a plug-in folder for each card, you cannot do this in LM DSR : because MDAPIFilter will not recognize the LM DSR process as one of its supported application)

Regards,
Stéphane.

[StephaneM]

Originally Posted by oaamaas

Tried the SmartRecorder on a WHS installation, SageTV 7.0.10 and dual Digital Everywhere Floppy DVB-T with CAM.../...Finally tried to add recorder to SageTV - fails

What do you see in the log files?

Usually failure to add the recorder to SageTV means that the recorder service failed to find the sage.properties file.

You can however add the recorder manually: in the LM DSR Config & Log folder (you can open it from the LM DSR taksbar notification icon) you'll find a Sage Tuner Properties.txt file.

Copy this file and edit it to replace:

$GUID$ by the ID of the recorder (you can choose any number not already used in SageTV)
$IP$ and $PORT$ by the IP address and TCP port of your recorder
$NAME$ by the name of your recorder

then copy / paste this edited file in your sage.properties file.

Regards,
Stéphane.

[owilsky]

Great! So I can finally get rid of DVBViewer...

Two questions:

1.
Who does the actual recording, SageTV? Or Smart Recorder like in the DVBViewer Recorder model?

If SageTV does the actual recording I have to do the configuration in sage.properties whether I want a .TS or an MPG-file?

2.
At the moment the channel scan only scans one transponder at a time? Is a full channel scan over all transponders planned?

Regards,
Oliver

[StephaneM]

Originally Posted by owilsky

Who does the actual recording, SageTV? Or Smart Recorder like in the DVBViewer Recorder model?

The recorder is doing the recording.

If SageTV does the actual recording I have to do the configuration in sage.properties whether I want a .TS or an MPG-file?

The recorder will only record in TS format. There is nothing to set up in SageTV, when you "Add To SageTV", it sets the format to TS automatically.

At the moment the channel scan only scans one transponder at a time? Is a full channel scan over all transponders planned?

Maybe with a .ini from KingOfSat.

Regards,
Stéphane.

[owilsky]

OK, I am way too stupid for this.
I don't get any channels on saying "Scan for new channels".
I want to scan transponder 71 on Astra 1H 19.2°E.
I am looking for the German channel "Das Erste" (FTA). Frequency is 11836.50 Horizontal which is MHz, so I have to enter 11836500 kHz. Polarity is horizontal, symbol rate 27500, FEC is 3/4, modulation QPSK. All of these values are from KingOfSat. Pilot and Roll Off are set to auto as they don't say me much.

I have only one digital quad LNB which is on a 4 -> 8 multiswitch. I am not sure if I need diseqc. I tried "none" and "Pos.A Opt.A" because this is the setting DVBViewer needed to get me a picture. I used the default "Universal LNB" and altered the settings for LNB.

Any idea?

[owilsky]

BTW: I think I found a bug: I found this line in my sage.properties after clicking on "Add to SageTV":

mmc/encoders/1262807007/100/0/encode_digital_tv_as_program_stream=$PSRECORD$

I guess the $PSRECORD$ shouldn't be there but a "false" for transport stream instead?

[StephaneM]

Originally Posted by owilsky

Frequency is 11836.50 Horizontal which is MHz, so I have to enter 11836500 kHz. Polarity is horizontal, symbol rate 27500, FEC is 3/4, modulation QPSK. All of these values are from KingOfSat. Pilot and Roll Off are set to auto as they don't say me much.

Seems ok to me:

Freq = 11836500
Pol = Horizonta
Sym. = 27500
FEC = 3/4
Modulation = QPSK
Pilot / RollOff = Auto (this is only used for DVB-S2)

If you made a little error on any of these values => it will not tune.

I have only one digital quad LNB which is on a 4 -> 8 multiswitch. I am not sure if I need diseqc. I tried "none" and "Pos.A Opt.A" because this is the setting DVBViewer needed to get me a picture. I used the default "Universal LNB" and altered the settings for LNB.

You should use the value you find in DVBViewer, I'm not sure if it will work. I verified Diseqc on my Technotrend hardware (TT USB S2-3650, with Toneburst A and Toneburst B and it works)

You should review the Live Trace Viewer when you do a scan.

Regards,
Stéphane.

[StephaneM]

Originally Posted by owilsky

BTW: I think I found a bug: I found this line in my sage.properties after clicking on "Add to SageTV":

mmc/encoders/1262807007/100/0/encode_digital_tv_as_program_stream=$PSRECORD$

I guess the $PSRECORD$ shouldn't be there but a "false" for transport stream instead?

You're right, it seems that I'm currently loading the wrong file (the DVBViewer Recorder one), I'll fix this.

Regards,
Stéphane.

[owilsky]

Now it worked... seems like going back (clicking "< Previous") to stop scanning (spinning circle on top right) and clicking again on "Next >" does not really work... but this is just a guess. BTW: "Pos.A Opt.A" worked for me now.

[antplugger]

Quick question, I am in Australia and use a couple of HVR-2200's for DVB-T and s-video capture from a set top box.

Could you please tell me the high level advantage I would get using this instead of standard SageTV recording as I do now ? Also would you run this on a different PC to the SageTV server

Thanks

[StephaneM]

Originally Posted by owilsky

Now it worked... seems like going back (clicking "< Previous") to stop scanning (spinning circle on top right) and clicking again on "Next >" does not really work...

Clicking "Next >" / "Cancel" / "Ok" or choosing another menu should cancel the scan (the scan times out after 25 sec. anyway), it does for me.

Anyway I uploaded a new version of the recorder (it is still version 1.0, but 1.0r2) with:

* Fix for the "Add to SageTV" feature
* Added the Diseqc support on Digital everywhere hardware (I forgot to include it in the first release)
* Added a new "Auto" settings for the size of the write buffer; because I have a little issue with SageTV 6 for SD shows SageTV pause the playback too much times if you are really close to the end of the file. So the "Auto" settings select a write buffer size according to the bit rate of the video you are recording (I have not tested it but it should work). Otherwise, if you usually watch HD Live TV you can set the write buffer size to 2MB, if you watch SD : 512 KB is ok, if you want to "watch" Live radio : 64 KB is the way to go (when you use a large buffer you use more memory, use less CPU and are easy on disk write)

Regards,
Stéphane.

[StephaneM]

Originally Posted by antplugger

Could you please tell me the high level advantage I would get using this instead of standard SageTV recording as I do now ?

Well, a little history:

* I created DVBViewer Recorder (my other recorder) because I'm using a CAM and the CAM wasn't working well with native SageTV support (and also the SageTV tuning was not that great on DVB-T)

* But I'm currently not so happy with DVBViewer that just stop recording with no clear indication of what is wrong (SageTV restart the recording but I loose 1 min of show... very annoying. I suppose this is related to a hardware issue as this is only happening with the Cinergy 2400i and never with the TT USB CT-3650, and even with my new recorder I see that the Cinergy is outputting sometimes a lot of wrong TS packet, for instance today was a nightmare because of the heat here in France the DVB-T reception was horrible : well my recorder still managed to record everything I asked, but the thing was unwatchable, I'm sure DVBViewer would have stopped the recording)

So I decided to write this new recorder where I can control 100% of the recording (Directshow was a nigthmare, especially when you do it 100% in managed code, even the filter I'm using to capture the DVB stream)

I had specific requirements to have a better solution than with DVBViewer:

* The recorder should highly optimized with the disk writing (I don't want to overflow my RAID 5 array so I had to create a buffered asynchronous disk writing engine). This is done and it works very well.

* The recorder had to extract subtitles in real time: I was already doing this in my previous recorder. This time the extraction is quicker as it is done when capturing and selecting the subtitle stream you want to extract is easier than with my previous recorder

* The recorder had to have an easy to use interface for defining lineups and scanning for channels. I wanted mostly a manual scan engine; because it's how I do it: I select the frequency I want to receive, do a scan, clean the list, proceed with another frequency : much easier for me than to scan many frequencies the one after another and sorting the hundreds of channels you get after... Then I also wanted the scan engine to be smart: here in France, the channel can be rearranged (frequency changes), so I wanted to be able to rescan a frequency and that the scan engine should be able to reconcile the channels automatically (so that hopefully a scan is the only thing to do to recover from a missing channel : that is not updating the XMLTV Importer or SageTV)

* The recorder also had to be smart when doing back to back recordings : I wanted that no show content was lost when SageTV stops the first recording and start the second one. So the recorder is using a "smart buffer" and whenever a recording stops, the recorder continue to record for a few seconds keeping the bytes received in a buffer, if a new recording starts on the same channel, the smart buffer content is retreived and appended to the new recording. This also works well and doesn't impact the SageTV scheduler (the smart buffer can be dropped by the recorder if needed to switch to another channel)

* I also wanted something very smooth to use, and I guess I suceeded in having a very fluid interface easy to use.

So the main benefit of this recorder is : the smart buffer and the disk write engine (SageTV doesn't use the same thing for recording neither in V6 and V7)

Also would you run this on a different PC to the SageTV server

You can run it on a different PC, this requires however to:

* be sure that you use UNC path in SageTV for your recording folder
* to change the account the LM DSR service is using (an account with admin rights and network access)

Though I don't know if the write engine I'm using will work that well over the network (You can disable asynchronous I/O and select a small write buffer size, so it may work).

I'm using the recorder on the SageTV server, it's safer : preformance is optimal and you will not suffer from any network issues (for instance DHCP IP changes which the recorder doesn't handle for now)

Regards,
Stéphane.

[rtengvad]

Hi Stephane

Great work! Any date on the DVB-C version?

Rasmus

[StephaneM]

Originally Posted by rtengvad

Any date on the DVB-C version?

No specific date yet, I'll wait some feedback on DVB-S and DVB-T for now (I can't test DVB-C at all where I am located) to see if anything major isn't hiding somewhere.

Regards,
Stéphane.

[JohanDW]

Hi Stephane,

Great news! Any information on performance compared to DVBViewer Recorder? I have played with that some time ago, but it was taking too long to switch from one channel to another one (at least 10 secs). Hope now that DVBViewer is out of the loop that things are faster....

Johan

[owilsky]

I am a real big fan of you I use your XML TV importer and DVB Viewer Recorder at the moment.

I played a bit with Smart Recorder and would really like to do some fieldwork test - to let it run for some days and see how it does.
But how do I connect it with EPG data? In DVB Viewer Recorder I could import a lineup from XMLTV importer. Is anything like this planned? What can I do until this is implemented?

Sorry for being such a pest, but I really would like to use this.

Oliver

[StephaneM]

Originally Posted by JohanDW

Great news! Any information on performance compared to DVBViewer Recorder? I have played with that some time ago, but it was taking too long to switch from one channel to another one (at least 10 secs)

I have not especially targeted speed, but essentially stability, so I would not expect this recorder to be faster than the DVBViewer one.

For instance:

* I always delay the playback in SageTV until suffiscient data are comitted to the disk (as for today, I wait until 2s of content are written to the disk, this may change in the future depending on how it perform here on my day to day use). If I don't do this, SageTV will reach the end of the file (that is the actual edge where content is being written to and this result in SageTV pausing during playback or worse reading erroneous data leading to continuity error during playback, so you have to pause or skip backward anyway)

* I destroy the Directshow graph when nothing is tuned anymore (If you choose no to do this (this is the "recycle after use" card option), at some point you'll have to restart the recorder service because you'll receive garbage or the recorder will not record anymore)

* I stop the graph on channel change (otherwise sometimes it doesn't tune, CAM is not decrypting etc...)

* I wait for channel to be fully decrypted before writing to the disk (something DVBViewer is not doing for instance: as a result when using a TT card with CAM there is many packets at the start of a recording that are still encrypted and this can freak out SageTV / DVBViewer...)

So you see, I have focused on stability, clean recordings and smooth playback when it does start in SageTV. The timings to expect are usually : 2 second for free to air and for channels that use CAM, well it depends on the speed of the CAM

here is a little example of how speedy it can get with an encrypted channel:

20100709 11:55:00.028 - Info - Starting new recording 'C:\Documents and Settings\HTPCUser\My Documents\My TV\TheFall-7478093-0.ts' on channel 'CANAL+ CINEMA'
20100709 11:55:00.028 - Info - Tuning to channel CANAL+ CINEMA (SID : 770) on USB 2.0 BDA DVB-T Tuner
20100709 11:55:00.028 - Info - Builing graph on USB 2.0 BDA DVB-T Tuner
20100709 11:55:00.231 - Info - TechnoTrend - OpenCI succeeded
20100709 11:55:00.231 - Info - Tuning to channel with SID 770 on frequency 522166 and bandwidth 8 on USB 2.0 BDA DVB-T Tuner
20100709 11:55:01.058 - Info - Running graph on USB 2.0 BDA DVB-T Tuner
20100709 11:55:02.431 - Info - Hertzien Payant - Recording started: C:\Documents and Settings\HTPCUser\My Documents\My TV\TheFall-7478093-0.ts
20100709 11:55:15.292 - Info - TechnoTrend - CAM is decrypting after 12 seconds of wait
20100709 11:55:15.292 - Info - Channel PMT : CANAL+ CINEMA (SID : 770)
                               PCR PID : 161
                               Stream #1(PID=161) : VideoMP4
                               Stream #2(PID=84) : AudioAC3, AC3 fra
                               Stream #3(PID=85) : AudioAC3, AC3 eng
                               Stream #4(PID=36) : DVBSubtitles, DVB Subtitles|fra|All aspect ratio|1|2 DVB Subtitles|fra|For Hearing Impaired, all aspect ratio|3|4

It takes one second to build the directshow graph, another second to run the graph (packets begin to flow through my code), but then you have to wait 12 seconds for the CAM to actually acknowledge the channel change (and a few more seconds until you finally receive you first clear video packet)

Regards,
Stéphane.