Recording to a network share using samba

[brzez]

My system and setup is as follows:

SageTV 6.3.8 running as a service on a WinXP SP2 machine using NextComm R5000 modded Dish 411 receiver and an HDTVFusion ATSC PCI tuner for local HDTV
2 HD100 Media Extenders
2.2 Terabyte Linux Ubuntu Samba Server running ZFS RaidZ2

I recently set up a network share using UNC paths. SageTV runs a service and writes to my linux Samba server. I've benchmarked my server and can write almost megabytes/sec to it sustained. Samba networking gives me about 3-4 megabytes/second write rates.

When recording content from Dish network through my R5000, recordings are saved to my network share and they play back just fine. LiveTV works well this way from Dish. My problem occurs when recording any digital channels using my FusionHDTV pci tuner. All of these recordings to the network share are pixelated and corrupt (verified by viewing the files through VLC). When recording to a local drive, the pixelation and corruption does not occur.

I'm trying to figure out what is different in these two cases. The dish bitrate never goes beyond 5 Mbps on the recordings while my FusionHDTV bitrate is somwehere between 10 and 17 Mbps for most HDTV recordings and between 2 and 10 Mbps for standard digital recordings. Are there tweaks for SageTV or Directshow to adjust buffering for recordings from my PCI Tuner? Could my Windows networking or Samba settings somehow be affecting things? Anyone else experience something similar?

Thanks..

-- John

[mikesm]

Originally Posted by brzez

My system and setup is as follows:

SageTV 6.3.8 running as a service on a WinXP SP2 machine using NextComm R5000 modded Dish 411 receiver and an HDTVFusion ATSC PCI tuner for local HDTV
2 HD100 Media Extenders
2.2 Terabyte Linux Ubuntu Samba Server running ZFS RaidZ2

I recently set up a network share using UNC paths. SageTV runs a service and writes to my linux Samba server. I've benchmarked my server and can write almost megabytes/sec to it sustained. Samba networking gives me about 3-4 megabytes/second write rates.

When recording content from Dish network through my R5000, recordings are saved to my network share and they play back just fine. LiveTV works well this way from Dish. My problem occurs when recording any digital channels using my FusionHDTV pci tuner. All of these recordings to the network share are pixelated and corrupt (verified by viewing the files through VLC). When recording to a local drive, the pixelation and corruption does not occur.

I'm trying to figure out what is different in these two cases. The dish bitrate never goes beyond 5 Mbps on the recordings while my FusionHDTV bitrate is somwehere between 10 and 17 Mbps for most HDTV recordings and between 2 and 10 Mbps for standard digital recordings. Are there tweaks for SageTV or Directshow to adjust buffering for recordings from my PCI Tuner? Could my Windows networking or Samba settings somehow be affecting things? Anyone else experience something similar?

Thanks..

-- John

There have been folks who have had issues with reliable writing to network volumes, but I have never seen a generic solution to these problems. The fact that SAMBA sucks so badly in your configuration makes me wonder what else is broken, and whatever that is may affect the recording as well.

Can anyone answer what sage does when it's attempting to write data coming from a tuner to a file and your get some error during the write? My network encoder apps like the R5000-HD just display and error and halt recording, but I can't tell what Sage itself does.

Can you try and run Netio (http://www.ars.de/ars/ars.nsf/docs/netio) benchmarks between the systems and see if there is a problem? I assume you are using Gigabit ethernet?

HD bitrates, even full ATSC rates at 19 mbps should not even cause a server to strain at all to handle.

[Sizam]

Hi John,
I also have a Solaris Express 3.7TB ZFS RAIDz2 fileserver and I'm using the built in CIFS (NFS/SMB) of RAIDz to share the volume. There is an issue with CIFS in that if you're reading AND writing to the volume at the same time you will get terrible to no performance. For instance, I can sustain a 70MB/s read OR write to the volume over GigaE but if I try to read AND write at the same time the write connection breaks off and the read finishes. For this reason I don't write directly to the share from SageTV since you would be performing read and writes.

Also, I highly suggest you stop using Samba and start using CIFS, I went from a 15-18MB sustained transfer rate to a 60-70MB/s sustained transfer.

Sam

[mikesm]

Originally Posted by Sizam

Hi John,
I also have a Solaris Express 3.7TB ZFS RAIDz2 fileserver and I'm using the built in CIFS (NFS/SMB) of RAIDz to share the volume. There is an issue with CIFS in that if you're reading AND writing to the volume at the same time you will get terrible to no performance. For instance, I can sustain a 70MB/s read OR write to the volume over GigaE but if I try to read AND write at the same time the write connection breaks off and the read finishes. For this reason I don't write directly to the share from SageTV since you would be performing read and writes.

Also, I highly suggest you stop using Samba and start using CIFS, I went from a 15-18MB sustained transfer rate to a 60-70MB/s sustained transfer.

Sam

Linux support for ZFS isn't the best right now. If you want to stay with linux you might want to go to RAID5 which is more proven, or just jump completely to Solaris and run ZFS and CIFS as Sam suggested. I advise people to try and stay in the sweet spot for OS's re: storage. These kinds of problems can be very tricky to solve, and can ruin a Sage system's performance and hit WAF badly.

[Sizam]

Yea,
Ditch Ubuntu, I had installed Ubuntu on an earlier iteration of my fileserver but Solaris RaidZ is where its at and Solaris was easy to pickup and install. The main downside of Solaris is driver support (or lack thereof), checkout the Solaris HCL to see if whatever controller card your using is supported.

Sam