HD Throughput Maxed Out?

[Taddeusz]

I'm wondering if I'm getting my hard drive througput maxed out. Here is my server config:

AMD X2 4200
2GB DDR 400
Asus A8N-E NF4 Ultra MB
Maxtor 300GB 16MB cache SATA 150 (40GB OS, the rest is video recording)
Seagate 500GB 7200.10 SATA 300 (single partition for video recording)
2x Hauppage PVR 150
HDHomeRun
Running 1 instance of ShowAnalyzer at 10% CPU usage
Dedicated PCI Intel Gigabit network card for the HDHR
All other networking is also Gigabit

Here is what I've observed and why I think I'm maxing out my HD throughput. I usually watch my HD recordings on my client PC. There are no problems I know of on it (C2D E6550, 2GB RAM, X1950XT, Cyberlink Decoder). I do occasionally get some annoying stutter on my client PC when playing only HD content though. When this is happening I notice the hard drive light on my server is solid or nearly solid. My ShowAnalyzer was using more CPU than it was supposed to but that problem has been rectified. I have noticed that when the stuttering is occuring most of the activity is happening on my Seagate drive. SageTV tends to record most things to that drive. I really notice it on nights when all four tuners are going. HD is unwatchable on my client then. However when very little to nothing is being recorded the HD plays back fine. CPU usage doesnt seem to be a problem. Even when it's recording 4 programs at once and processing a single recording for ads the CPU usage rarely gets above 40%

Is there any real solution to this problem? I've thought about RAID 0 but I really don't want the risk that goes along with that. I also have Intelligent Recording enabled. I've thought of disabling that so maybe it's not likely that it's going to be recording from all 4 tuners at once. Space obviously isn't a concern although I do have my PVR 150's record quality set to "DVD Standard Play" which is going to require more HD and PCI throughput.

I guess my real question is if there's any good way to figure out where my system is hitting a bottleneck?

Any suggestions would be great. I'm currently using an MVP on my TV and really looking forward to the HD extender. I'm little worried that I'll have the same problems I'm seeing on my client. I'd like to get this solved before then.

[gplasky]

Take a look at your 500GB drive and make sure the jumper is removed from it. If the jumper is still on it that will make your drive run at SATA150. You need to remove it to make sure it is running at SATA 300. That could be your bottleneck with 4 recordings going to the same drive and trying to watch another from that same drive.

Gerry

[Taddeusz]

I'm 100% certain that I removed the jumper when I upgraded my server to the NF4 board. I'll check in the NF SATA driver when I get home though.

I did some quick calculations and figured out that all the video recording requires at most 44.56Mb/s which is about 5.57MB/s. With that and playing back at most 15.88Mb/s plus ShowAnalyzer running at the same time I wonder if it's not the throughput that's the problem but seek latency that's killing it.

Oh, I forgot to mention that both my recording partitions are formatted with 64k clusters.

[gplasky]

I have the same 500GB drive (multiple drives) in my server and have had 2 HD, 2 SD recordings going to the drive and a HD recordings playing at a PC client and an MVP which requires transcoding and they played fine. I'm running comskip now but I know when I ran SA it could pound the drive pretty hard. Test it all without SA running and see if you have the playback problem.

Gerry

[Taddeusz]

I tried using the disk access limiting feature on ShowAnalyzer but what I thought was going to be a respectable speed turned out to be snail slow.

[evilpenguin]

Do you run SA on shows while they are recording or after they are done? If its the 2nd one, then its possible that SA is your problem.

I ran into almost the same issue on my server, but I found that if I limited comskip to 60 fps rather than letting it run as fast as it could the problem went away. 99% of the time i'm running my comskip on shows while they are recording and thus the speed is limited to 30/60 fps anyway so its not really a big deal.

[Taddeusz]

Originally Posted by evilpenguin

Do you run SA on shows while they are recording or after they are done? If its the 2nd one, then its possible that SA is your problem.

I ran into almost the same issue on my server, but I found that if I limited comskip to 60 fps rather than letting it run as fast as it could the problem went away. 99% of the time i'm running my comskip on shows while they are recording and thus the speed is limited to 30/60 fps anyway so its not really a big deal.

Yea, most of the time it's running on shows that are already finished recording. I was running two SA processes at a time but knocked it down to one so I think it spends all it's time processing stuff that's fully recorded.

SA doesn't have any kind of FPS limit. It has a CPU percentage limit and a disk access (in bytes per second) limit. I may try the disk limit again and see if I can get it working properly.

[GKusnick]

Originally Posted by Taddeusz

I guess my real question is if there's any good way to figure out where my system is hitting a bottleneck?

Use the performance monitoring tools built into Windows (Start > Administrative Tools > Performance). There are all kinds of metrics in there for disk, CPU, network, and more.

[lobosrul]

I don't think its your drive speed, but rather having different programs trying to access different chunks of it at once.

I have 2 tuners. And with both recording and one playback everything is fine. Two recordings and 2 clients playing back runs into problems if both shows are 1080i (higher bitrate). I currently have SA setup to run late at night. If I was only recording one show it was ok, but two (regardless if SA was set to only run one instance or two) + SA = stuttering.

The only way I think this could be fixed is if Sage allowed for a very large buffer. I have 1 GB of Ram on my client and 2 GB on my server (which is also sort of my 2nd client). Setting up a large (512MB?) buffer could probably help things.

[sandor]

Originally Posted by gplasky

Take a look at your 500GB drive and make sure the jumper is removed from it. If the jumper is still on it that will make your drive run at SATA150. You need to remove it to make sure it is running at SATA 300. That could be your bottleneck with 4 recordings going to the same drive and trying to watch another from that same drive.

Gerry

the jumper shouldn't really make a difference. SATA 150 hass a theoretical limit of about 150 MB/s (about 1.2 Gbps) - your single 500 GB hard drive will probably top out at about 1/2 of that. HD (OTA/ATSC) runs at a max of 2.3 MB/s (19 Mbps), so 4 ATSC feeds would max out at 9.2 MB/s - about what a hard drive could sustain back in 1998.

SATA150 or SATA300 won't matter - ATA100 could handle the throughput (1 TB of my current setup is on an ATA100 PCI card - no problems doing what you are doing).

first, check and make sure your recording drives are 64K partitions, this makes a huge difference with NTFS formatted drives. this is probably your problem, otherwise check teh SMART status on the drive and make sure everything is kosher.

[Taddeusz]

As I noted both my recording partitions have a 64K cluster size.

[Taddeusz]

This morning I tweaked my SA settings to limit disk acces to about 4MB per second. That's about 4 times real-time speed for my SD recordings. I can't remember what I set it to the last time I tried but I believe I really got it wrong. I think I had 4 digits in there instead of 7 thinking in KB/s rather than B/s. I hope this should be working well now. I'll tweak that up or down if I continue to have problems be it HD playback or ad detection performance.