Attempting something different... FreeBSD

[drewg]

Originally Posted by madtrader

This made me look into makemkv. Very interesting. I might have to give it a try, even. I use clownbd (and tools it relies on) to get HD200 compatible AC3 audio, though, so I have to use Windows eventually. Something makes me think you've already figured out around this as well.

Yes, I upgraded from an HD100 to an HD300, which is supposed to be able to handle BD audio formats.

Drew

[drewg]

Originally Posted by madtrader

I'm curious what your issue is with the HDHR. I too had horrible OTA reception, but I fixed that with the 2 huge antennae in the attic. I would only need one, but I think the local FOX station is VHF (it's been a while I might have that backward). I spent a couple evenings crawling around the attic getting things installed and adjusted. The VHF antenna is something like 10 feet across. Oh, and I've had 2 HDHR die on me, so I'm on my fifth.

Short story: The HDHR's tuner is not good for OTA with multipath.

Long story:

I've got 2 original CM4228s in my attic, one pointed at my local market's towers (15 miles away), and one pointed at the neighboring market (55 miles away).

My big problem is that I'm on a the back of a hill that is between me & my local towers, and I have a heavily wooded lot, so I have lots of multipath. I put my antennas up 6 years ago, and never adjusted them until I got my HDHR. Every recent tuner I have (all LGDT330x or Samsung based) get a lock with no errors at all on my local channels, except the HDHR which has breakups. It performs roughly as well as my circa 2004 nxt2002 based Air2PC card, which I retired years ago. I spent several evenings in the attic, and moved/adjusted the antenna, and managed to improve reception hugely on some stations (and worsen it marginally on others). It is sad that a $25 LGDT3305 based USB stick puts a $100 HDHR to shame.

I can get the 2 strongest stations from 55 miles away just fine with the HDHR. For those channels, the HDHR performs almost as well as my other tuners. But there is less (no?) multipath for them.. Unfortunately, I cannot get all the stations from my neighboring market, so I can't just switch to recording those channels.

Originally Posted by madtrader

The stutter is in the file being played back. The same file would play back fine if I wait for a point when fewer shows are recording simultaneously. In the

That's definitely the problem I'd prefer to have I had just the opposite problem under linux -- high write latency, combined with other activity (like browsing around in an extender's gui) would cause data overflow from my internal tuners, causing recordings that were glitchy whenever they were played.

Is ZFS prefetching enabled? Check sysctl vfs.zfs.prefetch_disable. I think it is disabled by default on ia32, and on amd64 if you have < 4GB RAM. For a playback problem with what's essentially sequential reads, prefetching sounds like it would help.

I'm curious about your h/w. Could you post (or PM me) your dmesg (assuming you've got one from right after booting).

Thanks!
Drew

[bagseed]

Originally Posted by madtrader

FreeBSD isn't really more complex. If you plan on running 64-bit Linux with the 32-bit JRE, it's probably about the same setup work.



I thought about this myself, but you lose some of the benefits of being on ZFS in the first place such as easy expansion of the pool and instant and consistent snapshots (with iSCSI you need to make sure the filesystem using the block device is clean (sync/unmount/whatever)). Maybe you aren't interested in or need those features though.



Nope, but not for the lack of trying. In the past I'd had trouble getting the installer. Sad excuse, I know. You have to watch for hardware support, though. Linux > FreeBSD > OpenSolaris in that regard. ZFS is more stable and feature full there if you have the hardware support, though.



Can you share where have you seen that? Is it that FreeNAS is moving to a FreeBSD 9.0-current base (which recently got v28), or is there some patch set back porting the work? I'd like to follow the development, but after some quick googling I don't find it.

In case anyone interested in ZFS hasn't seen this (it's gotten some decent coverage from a number of places recently) http://www.zfsbuild.com/ has some good coverage.

Hey bud, on the freenas.org page there is a zfs link.

http://www.freebsdnews.net/2010/09/0...upporting-zfs/

I heard it through word of mouth ....I think it will be implemented in 8.x via patches.


Since the sagetv server and storage server will be both VM's running in a Vmware HA / DRS cluster I figure it's pretty unlikely they will loose connection other then a straight up cluster failure for one reason or another. The storage sever will run in the shared storage under VMFS and I will just cut out raw hard-drive slices and attach them to the VM as needed. If I ever need to add more space I just add another drive to the vm and then add it to the pool via zfs tools. Also if I run out of physical space on the shared storage I can increase this pretty easily via vmwares tools and or build up a new unit and storage motion everything over. That's the theory anyways.

[madtrader]

Originally Posted by drewg

Yes, I upgraded from an HD100 to an HD300, which is supposed to be able to handle BD audio formats.

Hahaha. Reading the specs for the HD300, I thought I'd do the same, but I'm not going to replace all 3 of my HD200s just for that. Well, that and all the BD's I've already converted the audio on :P .

Originally Posted by drewg

Short story: The HDHR's tuner is not good for OTA with multipath.

Gotcha. I didn't realize they were that bad ... even with CM4228's.

Originally Posted by drewg

Is ZFS prefetching enabled?

# sysctl vfs.zfs.prefetch_disable
vfs.zfs.prefetch_disable: 1

Originally Posted by drewg

I'm curious about your h/w. Could you post (or PM me) your dmesg (assuming you've got one from right after booting).

Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-RELEASE-p2 #0: Tue Jan 12 00:58:18 CST 2010
    root@dagobah:/usr/obj/usr/src/sys/DAGOBAH
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Core(TM)2 Duo CPU     E6750  @ 2.66GHz (2660.01-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x6fb  Stepping = 11
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xe3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 8589934592 (8192 MB)
avail memory = 8255725568 (7873 MB)
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD 	 XSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci1
pci3: <ACPI PCI bus> on pcib3
atapci0: <Marvell 88SX6081 SATA300 controller> port 0x2000-0x20ff mem 0xd0200000-0xd02fffff irq 48 at device 1.0 on pci3
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
ata4: <ATA channel 2> on atapci0
ata4: [ITHREAD]
ata5: <ATA channel 3> on atapci0
ata5: [ITHREAD]
ata6: <ATA channel 4> on atapci0
ata6: [ITHREAD]
ata7: <ATA channel 5> on atapci0
ata7: [ITHREAD]
ata8: <ATA channel 6> on atapci0
ata8: [ITHREAD]
ata9: <ATA channel 7> on atapci0
ata9: [ITHREAD]
uhci0: <Intel 82801I (ICH9) USB controller> port 0x1820-0x183f irq 16 at device 26.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x2f00
usbus0: <Intel 82801I (ICH9) USB controller> on uhci0
uhci1: <Intel 82801I (ICH9) USB controller> port 0x1840-0x185f irq 17 at device 26.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x2f00
usbus1: <Intel 82801I (ICH9) USB controller> on uhci1
uhci2: <Intel 82801I (ICH9) USB controller> port 0x1860-0x187f irq 18 at device 26.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x0f10
usbus2: <Intel 82801I (ICH9) USB controller> on uhci2
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xd0000000-0xd00003ff irq 18 at device 26.7 on pci0
ehci0: [ITHREAD]
usbus3: EHCI version 1.0
usbus3: <Intel 82801I (ICH9) USB 2.0 controller> on ehci0
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci5: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci13: <ACPI PCI bus> on pcib5
em0: <Intel(R) PRO/1000 Network Connection 6.9.14> port 0x3000-0x301f mem 0xd0800000-0xd081ffff irq 16 at device 0.0 on pci13
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:30:48:be:ca:e2
pcib6: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
pci15: <ACPI PCI bus> on pcib6
em1: <Intel(R) PRO/1000 Network Connection 6.9.14> port 0x4000-0x401f mem 0xd0900000-0xd091ffff irq 17 at device 0.0 on pci15
em1: Using MSI interrupt
em1: [FILTER]
em1: Ethernet address: 00:30:48:be:ca:e3
uhci3: <Intel 82801I (ICH9) USB controller> port 0x1880-0x189f irq 23 at device 29.0 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x2f00
usbus4: <Intel 82801I (ICH9) USB controller> on uhci3
uhci4: <Intel 82801I (ICH9) USB controller> port 0x18a0-0x18bf irq 22 at device 29.1 on pci0
uhci4: [ITHREAD]
uhci4: LegSup = 0x2f00
usbus5: <Intel 82801I (ICH9) USB controller> on uhci4
uhci5: <Intel 82801I (ICH9) USB controller> port 0x18c0-0x18df irq 18 at device 29.2 on pci0
uhci5: [ITHREAD]
uhci5: LegSup = 0x2f00
usbus6: <Intel 82801I (ICH9) USB controller> on uhci5
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xd0000400-0xd00007ff irq 23 at device 29.7 on pci0
ehci1: [ITHREAD]
usbus7: EHCI version 1.0
usbus7: <Intel 82801I (ICH9) USB 2.0 controller> on ehci1
pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci17: <ACPI PCI bus> on pcib7
vgapci0: <VGA-compatible display> port 0x5000-0x50ff mem 0xd8000000-0xdfffffff,0xd0a00000-0xd0a0ffff irq 22 at device 4.0 on pci17
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <AHCI controller> port 0x1c50-0x1c57,0x1c44-0x1c47,0x1c48-0x1c4f,0x1c40-0x1c43,0x18e0-0x18ff mem 0xd0000800-0xd0000fff irq 17 at device 31.2 on pci0
ahci0: [ITHREAD]
ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: [ITHREAD]
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: [ITHREAD]
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich2: [ITHREAD]
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich3: [ITHREAD]
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich4: [ITHREAD]
ahcich5: <AHCI channel> at channel 5 on ahci0
ahcich5: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
pci0: <dasp> at device 31.6 (no driver attached)
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
cpu0: <ACPI CPU> on acpi0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
orm0: <ISA Option ROM> at iomem 0xc0000-0xc8fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
Timecounters tick every 1.000 msec
Waiting 5 seconds for SCSI devices to settle
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 12Mbps Full Speed USB v1.0
usbus6: 12Mbps Full Speed USB v1.0
usbus7: 480Mbps High Speed USB v2.0
ad4: FAILURE - SET_MULTI status=51<READY,DSC,ERROR> error=4<ABORTED>
ad4: 30533MB <OCZ VERTEX 1.4> at ata2-master SATA300
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <Intel> at usbus5
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen6.1: <Intel> at usbus6
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
ugen7.1: <Intel> at usbus7
uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
acd0: DVDROM <ASUS DVD-E616A3T/1.10> at ata3-master SATA150
ad8: 152627MB <Seagate ST3160815AS 3.AAD> at ata4-master SATA300
GEOM: ad8s1: geometry does not match label (255h,63s != 16h,63s).
GEOM_JOURNAL: Journal 4040654584: ad8s1e contains data.
GEOM_JOURNAL: Journal 4040654584: ad8s1e contains journal.
GEOM_JOURNAL: Journal 2006776180: ad8s1f contains data.
GEOM_JOURNAL: Journal 2006776180: ad8s1f contains journal.
GEOM_JOURNAL: Journal ad8s1e consistent.
GEOM_JOURNAL: Journal ad8s1f consistent.
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
uhub3: 6 ports with 6 removable, self powered
uhub7: 6 ports with 6 removable, self powered
acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 
(probe0:ata1:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 
(probe0:ata1:0:0:0): CAM Status: SCSI Status Error
(probe0:ata1:0:0:0): SCSI Status: Check Condition
(probe0:ata1:0:0:0): NOT READY asc:3a,1
(probe0:ata1:0:0:0): Medium not present - tray closed
(probe0:ata1:0:0:0): Unretryable error
(aprobe0:ahcich0:0:15:0): SIGNATURE: 0000
(aprobe0:ahcich0:0:0:0): SIGNATURE: 0000
(aprobe1:ahcich1:0:15:0): SIGNATURE: 0000
(aprobe0:ahcich1:0:0:0): SIGNATURE: 0000
(aprobe2:ahcich2:0:15:0): SIGNATURE: 0000
(aprobe0:ahcich2:0:0:0): SIGNATURE: 0000
(aprobe3:ahcich3:0:15:0): SIGNATURE: 0000
(aprobe0:ahcich3:0:0:0): SIGNATURE: 0000
(aprobe4:ahcich4:0:15:0): SIGNATURE: 0000
(aprobe0:ahcich4:0:0:0): SIGNATURE: 0000
(aprobe5:ahcich5:0:15:0): SIGNATURE: 0000
(aprobe0:ahcich5:0:0:0): SIGNATURE: 0000
ada0 at ahcich0 bus 0 target 0 lun 0
ada0: <Hitachi HDS722020ALA330 JKAOA28A> ATA/ATAPI-8 SATA 2.x device
ada0: 300.000MB/s transferscd0 at ata1 bus 0 target 0 lun 0
cd0: <ASUS DVD-E616A3T 1.10> Removable CD-ROM SCSI-0 device 
cd0: 3.300MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed

ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: Native Command Queueing enabled
ada1 at ahcich1 bus 0 target 0 lun 0
ada1: <Hitachi HDS722020ALA330 JKAOA28A> ATA/ATAPI-8 SATA 2.x device
ada1: 300.000MB/s transfers
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: Native Command Queueing enabled
ada2 at ahcich2 bus 0 target 0 lun 0
ada2: <Hitachi HDS722020ALA330 JKAOA28A> ATA/ATAPI-8 SATA 2.x device
ada2: 300.000MB/s transfers
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2: Native Command Queueing enabled
ada3 at ahcich3 bus 0 target 0 lun 0
ada3: <Hitachi HDS722020ALA330 JKAOA28A> ATA/ATAPI-8 SATA 2.x device
ada3: 300.000MB/s transfers
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3: Native Command Queueing enabled
ada4 at ahcich4 bus 0 target 0 lun 0
ada4: <Hitachi HDS722020ALA330 JKAOA20N> ATA/ATAPI-8 SATA 2.x device
ada4: 300.000MB/s transfers
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4: Native Command Queueing enabled
ada5 at ahcich5 bus 0 target 0 lun 0
ada5: <Hitachi HDS722020ALA330 JKAOA20N> ATA/ATAPI-8 SATA 2.x device
ada5: 300.000MB/s transfers
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5: Native Command Queueing enabled
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/ufsid/4a96cb0420c147b6
ZFS filesystem version 13
ZFS storage pool version 13
em1: link state changed to UP
eth0: link state changed to UP

An extra little bit for completeness:

dagobah# zpool status pool
  pool: pool
 state: ONLINE
 scrub: scrub completed after 5h54m with 0 errors on Mon Oct 25 15:48:46 2010
config:

	NAME                STATE     READ WRITE CKSUM
	pool                ONLINE       0     0     0
	  mirror            ONLINE       0     0     0
	    label/hitachi0  ONLINE       0     0     0
	    label/hitachi1  ONLINE       0     0     0
	  mirror            ONLINE       0     0     0
	    label/hitachi2  ONLINE       0     0     0
	    label/hitachi3  ONLINE       0     0     0
	  mirror            ONLINE       0     0     0
	    label/hitachi4  ONLINE       0     0     0
	    label/hitachi5  ONLINE       0     0     0

errors: No known data errors
dagobah# zfs list
NAME           USED  AVAIL  REFER  MOUNTPOINT
pool          4.26T  1.09T    18K  none
pool/media    4.18T  1.09T  4.18T  /var/media
pool/opt      88.1G  1.09T  88.1G  /opt

[madtrader]

Originally Posted by bagseed

Hey bud, on the freenas.org page there is a zfs link.

http://www.freebsdnews.net/2010/09/0...upporting-zfs/

I heard it through word of mouth ....I think it will be implemented in 8.x via patches.

Ah. This announcement I'd seen before. I thought you were referring to something specific about v28 patches being integrated into FreeNAS soon.

Originally Posted by bagseed

Since the sagetv server and storage server will be both VM's running in a Vmware HA / DRS cluster I figure it's pretty unlikely they will loose connection other then a straight up cluster failure for one reason or another. The storage sever will run in the shared storage under VMFS and I will just cut out raw hard-drive slices and attach them to the VM as needed. If I ever need to add more space I just add another drive to the vm and then add it to the pool via zfs tools. Also if I run out of physical space on the shared storage I can increase this pretty easily via vmwares tools and or build up a new unit and storage motion everything over. That's the theory anyways.

Wow, sounds like quite a setup. I've not played with iSCSI before -- so you can easily add storage to the pool backing the iSCSI volume, but how do you grow the iSCSI volume? Can you do it without taking it offline? I imagine there's quite some dependency on the filesystem you put on it.

[drewg]

Originally Posted by madtrader

Hahaha. Reading the specs for the HD300, I thought I'd do the same, but I'm not going to replace all 3 of my HD200s just for that. Well, that and all the BD's I've already converted the audio on :P .

It was an easier decision for me -- one HD100 was certainly worth the upgrade! Although, I have not ripped any BDs since upgrading. The BDs tend to come clustered around my son's birthday, and Christmas .

Originally Posted by madtrader

# sysctl vfs.zfs.prefetch_disable
vfs.zfs.prefetch_disable: 1

That's the first thing I'd try tweaking (set it to 0 to enable prefetching).

Drew

[troll5501]

Originally Posted by madtrader

I've hit a new bottleneck -- recording 5 or 6 HD streams (it varies) while attempting to watch an OTA HD recording stutters. In looking at the I/O stats for the drives (via gstat, in case anyone wanted to know), it appears I need to do some ZFS tuning. I get the bursty write action.

I'm using ZFS for my Sage storage via SAMBA and NFS. I ran into the same bursty write issues and it took some time (and some more recent ZFS code fixes) to get it all straightened out. I'm using OpenSolaris/OpenIndiana so I can't provide specifics about tuning the kernel parameters in FreeBSD, but here are some things that you can try:

- Reduce zfs_vdev_max_pending from 35 to a smaller value (10 is now the default in the latest ZFS code)

- Reduce zfs_txg_synctime (now called zfs_txg_synctime_ms and defaults to 1 second vs 5 seconds in the older ZFS code) and/or zfs_txg_timeout -- this will help sync the transaction groups to disk more frequently so the "batches" are smaller

- Experiment with zfs_write_limit_override -- it should allow you to smooth out the writes by reducing the amount of data written to disk in a single "batch". With this, you may not need to change the synctime/timeout setting.

- If you enable prefetch you may want to consider reducing the size of the prefetch operation. I found that during FF and REW operations in Sage (especially 64x and 256x), the default prefetch size was causing ZFS to read many times more data from disk than what Sage itself was requesting, causing very busy disks. Default value for zfetch_block_cap is 256 (blocks). I'm using 64.

[madtrader]

Originally Posted by drewg

That's the first thing I'd try tweaking (set it to 0 to enable prefetching).

I actually set that myself. I ran into issues fast forwarding like Troll5501 mentioned. Only an issue while recording multiple shows while playing back another.

Originally Posted by Troll5501

... here are some things that you can try ...

Thank you. A couple of these I was already looking at. As I mentioned, I have some set to take affect for my next reboot. Specifically, here are the changes (set in /boot/loader.conf):

vfs.zfs.prefetch_disable="1"
vfs.zfs.txg.timeout="5"
vfs.zfs.vdev.max_pending="32"

Maybe I should lower vfs.zfs.vdev.max_pending further. 32 came from reading that's the size of the AHCI queue size. It doesn't appear zfs_write_limit_override is exposed for tuning on FreeBSD 8.0. I've found a patch that does expose it. I'll start with these other tunables, but try that if I'm not successful. I wonder if FreeBSD 8.1 has it exposed.

Maybe I should re-enable prefetch and adjust zfetch_block_cap first. I have 8GB of RAM, so prefetch really ought to be enabled.

[madtrader]

All this talk has spurred me into action. I've done some ZFS tuning and the results are success. Here are the pertinent parts of my /boot/loader.conf now:

#vfs.zfs.prefetch_disable="1"
vfs.zfs.txg.timeout="5"
vfs.zfs.vdev.max_pending="10"
vfs.zfs.zfetch.block_cap="64"

My first change was to set vfs.zfs.txg.timeout to 5 and vfs.zfs.vdev.max_pending to 10. That change alone was enough to allow me to simultaneously record 6 HD streams and play another one back without stuttering. I'd prefer to have the prefetching enabled again, so I decreased the vfs.zfs.zfetch.block_cap to 64. With all these settings I can still simultaneously record 6 HD shows while playing back another. I even tried it playing back a ripped Blu-Ray -- no stutter at all. Even with the prefetch block cap decreased, I can see that reads are heavier during fast forwards, but it's still low enough for these drives to handle it.

[troll5501]

Originally Posted by madtrader

With all these settings I can still simultaneously record 6 HD shows while playing back another. I even tried it playing back a ripped Blu-Ray -- no stutter at all. Even with the prefetch block cap decreased, I can see that reads are heavier during fast forwards, but it's still low enough for these drives to handle it.

That's great news! Glad to hear that you've got everything working now.