HDTV and Hard Drives

[BBCritical]

Would there be a decrease in stuttering if you used a SATA 3.0 hard drive to record to as opposed to a PATA drive? And also, does a IDE to sATA Converter knock the transfer speed of a SATA 3.0 drive back to s PATA speed?

[sjgore]

Benchmarks have proven that SATA300 speeds are nowhere near twice as fast as SATA150 speeds (in real-world use), but SATA150 is definitely quicker than PATA/IDE.

If you want faster hard disks, I would highly recommend a RAID-0 array, with a 1024k striping size if to be used entirely for video, otherwise a 32k striping size. I currently have a 1TB RAID-0 array consisting of 2x 500GB Western Digital SATA300 hard disks set to a 32k stripe size. It's very fast!

Steve.

[BBCritical]

But will that decrease stuttering in HDTV recordings?

[sjgore]

Well, assuming that the stuttering is caused by a bottleneck somewhere in the system (due to the large data transfer rates and processing power needed for HD), it will certainly mean that your hard-disks are not the bottleneck.

I don't know much about the stuttering problem, since I don't experience it myself, but your post implied that hard disk speed might be causing it. Apologies if I've misunderstood!

Steve.

[paulbeers]

Are you sure your 9550's aren't the issue with the studdering? Are you trying to transcode HDTV for your mvp's? Traditionally studder isn't created by the hard drives. Even at 9 GB per hour recording, only works out to about 2.56 MB/s. That is well within any fairly new hard drive's capicity. If you are playing and recording multiple streams, that could pose some problems, but I can't see that being much of a problem if you are outputing to 2 different HDTV clients at a time and recording (still would be under 10 MB/s although I realise the multiple Read/Write instructions would slow the hard drives output).

While I haven't tried much more than recording on my server and then playing onto one of my clients at a time, all my studder was due to poor drivers for my vid cards (sometimes you just have to find the sweet driver for your card) and different codecs. Cleaned up my studder. I also only use Overlay.

[BBCritical]

Maybe it is the video card or the drivers. I will try different ones when I get home ... hopefully I dont need to upgrade video cards because I love the HD... If I had hd extenders (when they come out) would this issue go away if everything recorded correctly?

[paulbeers]

Theoretically yes. When you are seeing studdering, does it make any difference how many of your clients are being used? If it is a disk issue, then the more clients you have that are accessing records, the worse it would be. If the studdering isn't marginally worse, then it is the video card. My vid cards are about as minimal as you can get and in fact I run them only at 720P just to minimize the stress on them. I am waiting for the HD extenders. I don't want to buy several high end cards (spend 150 bucks) and right around the corner I can spend 180 for an HD extender and not have to worry about fixing software issues. I love the MediaMVP.

[teknubic]

I doubt that the harddrive is the bottleneck. I have a single 250GB PATA recording drive in a server that heavily favors the HDTV tuners and I don't have stuttering at all.
Here are theoretical speed times just for reference:
UltraSCSI - 2560Mbit/s
eSATA - 2400Mbit/s
SATA300 - 2400Mbit/s
PATA133 - 1064Mbit/s
FireWire800 - 786Mbit/s
USB2.0 - 480Mbit/s
FireWire400 - 400Mbit/s
USB1.0 - 12Mbit/s

[SHS]

teknubic your number are a bit off on PATA133 - 1064Mbit/s when it should be 133Mbit/s that is the max theoretical speed but in real world it more like avg out a round 30 to much as 60 in non brust mode

[teknubic]

Actually, they're correct. You are confusing Mbit/s with Mbyte/s. ATA133 is 133 MegaBytes per second. There are 8 bits in a byte so 133MByte/s x 8 = 1064Mbit/s
But you are correct -- real world tests are way lower. I was just providing the theoreticals.

[SHS]

Yup I forgot the B vs b thing

[Goodspike]

Originally Posted by teknubic

I doubt that the harddrive is the bottleneck. I have a single 250GB PATA recording drive in a server that heavily favors the HDTV tuners and I don't have stuttering at all.
Here are theoretical speed times just for reference:
UltraSCSI - 2560Mbit/s
eSATA - 2400Mbit/s
SATA300 - 2400Mbit/s
PATA133 - 1064Mbit/s
FireWire800 - 786Mbit/s
USB2.0 - 480Mbit/s
FireWire400 - 400Mbit/s
USB1.0 - 12Mbit/s

What I will add to this discussion is this as it pertains to HD recordings. On my system, with either a SATA or PATA drive, as the size of the file gets larger, the CPU utilization increases dramatically. It increased 10-20% (so say from 50% to 60 to 70%) as the file gets to be 1.5 hours long. But it's only a problem while recording the file and simultaneously watching it. If I'm recording such a file, and simultaneously watching some other file, the increase doesn't occur.

It's as if Windows has an issue dealing with the simultaneous writing and reading of such large files. So it's not a throughput issue, because in both cases the throughput would be the same.

It may be a Sage issue, because I've noticed that CPU utilization is higher if you have SageService running than if you don't. There again, it's not a throughput issue, because the amount of data transferred is exactly the same.

[appelm]

I've been wondering about this too. I just got an HDhomerun last week and have been having some stuttering issues with HD recordings. The thing is its definitely on the recording side and not the playback side. If I open the actual .ts file on another machine and in any other player (like vlc player), I can replicate the stutter in exactly the same spot in the recording. I think this means something is going wrong during recording. I've found it happens regardless of 1 tuner or 2 tuners in use. The recording will playback fine for 2-3 minutes, and then there will be a glitch where it seems to skip forward several seconds (to loud poping sounds) and then continue playing normally without stutters for several more minutes before coming upon another glitch. Any thoughts?

Here's my hardware:
Server:
p4 2.4ghtz, 512mb ram
3 IDE pata maxtor drives, 1 500 and 2 250's.
Sage 6.1.3 (going to try 6.1.4 shortly)
hdhomerun over gigabit lan

Thanks,
Marc

[appelm]

Originally Posted by appelm

I've been wondering about this too. I just got an HDhomerun last week and have been having some stuttering issues with HD recordings. The thing is its definitely on the recording side and not the playback side. If I open the actual .ts file on another machine and in any other player (like vlc player), I can replicate the stutter in exactly the same spot in the recording. I think this means something is going wrong during recording. I've found it happens regardless of 1 tuner or 2 tuners in use. The recording will playback fine for 2-3 minutes, and then there will be a glitch where it seems to skip forward several seconds (to loud poping sounds) and then continue playing normally without stutters for several more minutes before coming upon another glitch. Any thoughts?

Here's my hardware:
Server:
p4 2.4ghtz, 512mb ram
3 IDE pata maxtor drives, 1 500 and 2 250's.
Sage 6.1.3 (going to try 6.1.4 shortly)
hdhomerun over gigabit lan

Thanks,
Marc

Any thoughts on this HD bandwidth question? Also, do you think this pc has enough juice to support hd?

[iammike]

I've only been using Sage for a few months now, but I've been recording HD to PC hard drives for several years with MyHD cards. I have a variety of hard drives I record to without problem. The main recording drives in my PC's are Sata 150, but I've also recorded to a PATA drive in an external USB attached enclosure without problem. With the SATA's I can record one program while streaming another to a second PC. In the Sage environment, I can watch live HD from an R5000HD modded 4dtv receiver without problem.

The biggest problem I've had with Sage and stuttering is codec and video card related, but once I started running the NVidia Purevideo codec they went away. I also picked up a 7600GT card which works perfectly, but my original 6800 worked fairly well too. Since you're thinking the problem is on the recording side, I'd be suspicious of signal strength. I've seen issues like you mention with OTA broadcasts and also satellite when the signal strength or quality gets too low. I don't know anything about the HDHomerun so I don't know if that could be the case here or not.

[jkohn]

Drive fragmentation can play a big part. We tend to record a few shows every night, watching them and then deleting them. After a while the drive can get very fragmented. Recording multiple HD streams to a fragmented drive while also watching something can cause dropouts on even the fastest hard drives. Since setting up a scheduled task to defragment my recordings drive every morning, I have far fewer problems with errors in recordings.

[phelme]

My big problem has been the NBC show Friday Night Lights; not the content of course, love the show, but the pixelated, stuttering mess it's been the last few weeks. This weekend I've been trying to figure out why that HD show and *only* that HD show ever has stuttering problems (Heroes, 24, Studio 60, Lost, Raines etc. never have problems). Well, near as I can tell, it's the simultaneous recording of other shows at the *same time* that chunkify FNL. here is what I've made note of:

- Bones (FOX 720P), Friday Night Lights (NBC 1080i) and The Daily Show (ComCent NTSC low-fi) are all recording at the same time
- Bones & FNL coming from an HDHomeRun box, Daily Show from a Hauppauge USB2 PVR
- all 3 shows coming into server via Ethernet
- all 3 shows recording to the same drive, even though 2 SATA, 64K block formatted drives are available... Jeers to Sage's not automatically handling this better (yes, yes can be directed via the properties file, but still..)
- it's the recording of FNL, not the playback that's at issue, as it looks like crap no matter what decoder, player, nor machine it's played on
- CPU is never taxed during any of this

On closer inspection, Bones *is* stuttering/pixelating a tiny bit. Ah ha! Since it's a progressive show though, the decoders don't choke on it quite so much as having to reassemble an interlaced show like Friday Night Lights.

the problem could be a couple of things:
- we're writing out 3 shows, 2 of which are of the heavy duty HD type, to a single spindle, at the same time. that's taxing the drive.
- Fragmentation City, Arizona (nod to Fernwood 2-Night ), later discovered some of the HD files had 5000+ fragments... yikes!
- they're all coming in through the network at the same time. it is possible, though I think unlikely since we're using 100 Mbps Ethernet, that packets are getting dropped.
- all this, drive & network, is being handled by the motherboard chipset, though I doubt that's an issue
- my BIOS drive settings are messed up

here's how I'm going to address the issue, all combined I hope this fixes it:
- installed PefectDisk to defragment the drives on a daily basis (both drives were 40% fragmented!)
- use forced_video_storage_path_prefix in the Sage.properties file to force tuners to particular drives, this way the HD shows will not be written to the same drive at the same time

I won't address the network at this point as I hope addressing the drives fixes it. still need to double check my BIOS to see if there are any issues there.

I'm now wondering whether an HDHomeRun solution vs. a server-localized HD solution (i.e. skipping the network) isn't such a good idea too. If addressing the drives doesn't entirely fix it, I'll try swapping out the 10/100 hubs I have in place for switches.

I'll know later this week if this fixed the problem and I'll post an update.

[phelme]

well, after defragging both drives and changing the target for the mpg files so the data is spread out over the spindles, there is still an issue.

I recorded Bones (720P) & Good Eats (480i) to one drive and ANTM (1080i) & Friday Night Lights (480i) to another simultaneously. Both HD shows, Bones & ANTM still show blocky pixelation and drop outs in the recorded file. Rats.

Since I wasn't there to observe this in person I ran another test late last night and Recorded Letterman (1080i), Leno (1080i) & whatever was on ESPN (480i) all simultaneously. This time, everything was clean, no blockiness, no pixelation. And interestingly enough since it was late at night, comskip started doing its thing around the same time and didn't make things worse. though it reads from disk rather than writes so has less of an impact.

Now I'm really confused.

I'm not sure what to make of all this. Obviously I need to record a 720p, & 1080i program along with some two 480i's at the same time to see if I can duplicate the exact problem from earlier. But the inconsistency of it all has my puzzled. Maybe the secret lies in the stream rate of the shows themselves. But Both ANTM (which isn't true HD anyway) and Bones are in the 19 Mbps range, hardly taxing.

Maybe it's the HDHomeRun itself??

Anyone else out there with multiple HD sources see pixelation problems like this in their recordings?

[phelme]

ok, i'll just keep talking to myself here. doesn't bother people on the subway.

another interesting data point I discovered. at the 30 (of 60 total) minute mark on the corrupt HD shows the drop outs & plixelated crap just stops. yes, you guessed it. when the fourth 30-minute show stops recording and we're down to just three (2 HD, 1 SD) simultaneous shows (of 60 min. each), everything records just dandy. So something, the motherboard bus, disk cache, network buffer, drive throughput, something is being pushed to the limit when 2 (HD) + 2 (SD) shows are recording at once. still sucks, but at least it's another clue.

I guess it could be my aging ECS motherboard or I just have to face facts and start looking at RAID.

[jkohn]

Originally Posted by phelme

I guess it could be my aging ECS motherboard or I just have to face facts and start looking at RAID.

I was having similar problems when recording 4 sumultaneous HD programs (and sometimes even 3). Contrary to the conventional wisdom, I've found that putting my two recording drives in a RAID-0 array gives me better performance/reliability than keeping them as two separate drives. Regular defragmentation also helps. I still get the occasional hiccup here and there, but nothing regular and I think these could have to do with the cable/OTA reception or the network source itself. But they're pretty rare now so I'm happy.