Help! Building new SageTV Server / Media Server

[matterw]

So, I have decided that it is time to upgrade my server -- the bug has arrived!

Currently setup:
* SageTV 6.6.2
* OS: Windows XP
* Mobo: Intel D865PERL
* CPU: P4 2.8Ghz
* Memory: 1GB
* Add-In: 3Ware 9500S-4LP (4port) [Running Raid5]
* Storage: 4 x 1TB
* Video Capture: 1-HD PVR, 1-WinTV (23xxx) dual tuner
* Case: <a joke>
* PSU: <a joke>

For new setup looking at:
* SageTV 6.6.2 for WHS
* OS: WHS (headless)
* Mobo: ???
* CPU: Xeon?
* Memory: ???
* Add-In: ???
* Storage: 4 x 1TB (Would like to add 10-16 more drives)
* Video Capture: 2-HD PVR, 1-WinTV (23xxx) dual tuner
* Case: Lian-Li <???>
* PSU: Corsair HX1000W
[Note, price range around $200-$600 for mobo+cpu]

So, I have been looking around at other people's setups and my head is spinning. Here are just two examples: 1) When I look at Skirge01 he has chosen a Super Micro Xeon board and 2) when I look at Savage1701 he has chosen an Asus mother board. Both look kick a**....

My system now has taken me so far, but get slow and I have noticed that it has slight pauses from time to time. It is used for SageTV plus media server not backup). I want something that is over powered. I do not want it to have any issues.

I'm leaning towards a Xeon CPU and using the WHS pooling (no raid) as my general guideline. Looking into other "unraid" solutions like FlexRaid, but don't know enough yet.


Can you provide some next steps on mother boards (SuperMicro, Asus, Gigabit, etc...), CPUs and Add-In cards to start with and why?

[gplasky]

Your best choice for motherboard type will be the ones that also provide drivers for Windows 2003 server. WHS is based on W2k3 server and having specific drivers for that OS will benefit you in the long run. For tuner cards I doubt you'll come across any W2k3 server drivers for those but you will need Windows XP drivers for that. (Not Vista or Win 7) I'm not sure you require the expense of a Xeon processor. A good, speedy quad processor will do. Sage doesn't need it unless yo have to do transcoding on the server. Other processes external to Sage like Comskip will also benefit from the quad.

Gerry

[matterw]

Originally Posted by gplasky

...I'm not sure you require the expense of a Xeon processor. A good, speedy quad processor will do. Sage doesn't need it unless yo have to do transcoding on the server. Other processes external to Sage like Comskip will also benefit from the quad.

Follow ups:
1) Curious why you suggest not needed the Xeon. If I put 2-3 of the HD-PVRs won't they need CPU power to transfer data via USB and do some work?

2) I am going to select an Intel processor -- like the brand. I know AMD is a good processor too, but Intel has treated me right so far. So, now that I am settled on Intel (like that has narrowed things) is there a suggestion on the Quad processor or family (i.e., Core 2-Duo) I should be looking at?

-Matt

[davephan]

One of the first decisions is which operating system you plan to use. WHS is great for drive expansion and redundancy, but may fall short to recover itself without an image, using a slow and painful scratch re-build. Windows XP Pro will work, but the support will probably drop in a year or two. Vista has it's problems but some people use it for SageTV. Windows 7 might be a choice too. You also have to choose between 32 or 64 bit.

I am thinking about rebuilding my SageTV computer with a GIGABYTE GA-EX58-UD5 LGA 1366 Intel X58 ATX Intel Motherboard mainboard

http://www.newegg.com/Product/Produc...82E16813128362

This mainboard is a 1366 and DDR3 memory, both cost more than older technology. People complain about the 1X PCI-E location making it worthless, the SATA connectors, the wrong memory locations to use in the manual, and many other things. Everything seems to have some bad reviews at Newegg.

I would use an Intel quad core instead of a Xeon, with a clock speed of 3 gigs if you can afford to go a bit faster. I'm not sure why you would need a Xeon, since the performance is usually less and the cost is higher.

The Lian-Li is a good brand. I wanted a full-tower black case, not a case with see though windows. I have a Corsair 850, and I think a Corsair 1000 watt is a good choice, except the price really jumps when you get to 1000 watts.

You might think about using Thermaltake A2309 iCage 5.25" bay convert to 3 x 3.5" HDD Modules for mounting additional drives. I could bolt in quite a few of these modules in my full-tower case.

http://www.newegg.com/Product/Produc...82E16811999141

If you are using 1 - 2 TB drives, you're talking about a lot of storage. Long before I got to 6 TB storage, I have the problem where most of my videos never get watched, since no one has that much time at my home. It's tough to delete things you want to save, but maybe you don't need to go that high with storage. With that being said, I bet my storage will probably double in within a couple years to 12 TB!

Memory. You'll have to decide if you want to stay below or go about 4 gigs. Some of the tuner cards won't work with more than 4 gigs. The newer cards generally will.

You might think about adding a HDHomerun for over the air HD channels. The HD-PVR can be used for other channels. Partitioning the channels will reduce the conflicts, and maybe you could get by with only one HD-PVR and one less monthly set top box rental fee.

I don't see any HD-200 units on your list. If you are using MVP units, replace them with HD-200 units. I plan to replace my MVPs on my main TVs and move them to infrequently used TVs.

If you can, you should also include a recovery plan from the beginning with your new system, such as disk imaging or some other method. If the system fails, you'll want to be able to recover it quickly with minimum downtime and hassle.

Dave

[gplasky]

Originally Posted by matterw

1) Curious why you suggest not needed the Xeon. If I put 2-3 of the HD-PVRs won't they need CPU power to transfer data via USB and do some work?-Matt

No-not really. All the encoding that is done for the HD-PVR is done with hardware on the device. Recording is the least cpu intensive process on a SageTV server. A Xeon is not your best bang-for-the-buck cpu out there. Unless you already have one in hand. Example:
Intel Core 2 Quad 2.83 GHz Processor 95W - $153 new

Intel Quad-Core Xeon 2.4 GHz Processor 105W - $210 new

Originally Posted by matterw

2) I am going to select an Intel processor -- like the brand. I know AMD is a good processor too, but Intel has treated me right so far. So, now that I am settled on Intel (like that has narrowed things) is there a suggestion on the Quad processor or family (i.e., Core 2-Duo) I should be looking at?

-Matt

As far as bang-for-the-buck look for the fastest, lowest-price Core 2 Quad. The pnce mentioned above is a good deal. Then you have a Intel Core i5-750 Lynnfield 2.66GHz 8MB L3 Cache LGA 1156 95W Quad-Core Processor for around $200. Where a Intel Core i7 940 LGA1366 2.93 GHz 8MB 45nm 130W Quad Core Desktop Processor goes for about $590. (Or a Intel Core i7-920 Bloomfield 2.66GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1366 130W Quad-Core Processor for $290).

Then you may also want to consider wattage used by the processor.

Gerry

[matterw]

Originally Posted by davephan

You might think about using Thermaltake A2309 iCage 5.25" bay convert to 3 x 3.5"

Thanks -- this looks like a good item for the cost!

Originally Posted by davephan

I don't see any HD-200 units on your list. If you are using MVP units, replace them with HD-200 units. I plan to replace my MVPs on my main TVs and move them to infrequently used TVs.

I should have included this: currently own 2 HD-200's and 1 HD-100

Originally Posted by davephan

I would use an Intel quad core instead of a Xeon, with a clock speed of 3 gigs if you can afford to go a bit faster. I'm not sure why you would need a Xeon, since the performance is usually less and the cost is higher.

There are a lot of choices as you are aware. Can you provide the top 2 or 3 Intel Quad processors you would recommend and why?

[matterw]

Originally Posted by gplasky

As far as bang-for-the-buck look for the fastest, lowest-price Core 2 Quad. The pnce mentioned above is a good deal. Then you have a Intel Core i5-750 Lynnfield 2.66GHz 8MB L3 Cache LGA 1156 95W Quad-Core Processor for around $200. Where a Intel Core i7 940 LGA1366 2.93 GHz 8MB 45nm 130W Quad Core Desktop Processor goes for about $590. (Or a Intel Core i7-920 Bloomfield 2.66GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1366 130W Quad-Core Processor for $290).

So, if I am looking at the Core 2 Quad Processors...
1) is there a big difference (based on my needs for a media server) between Intel 8200, 8300, 9400, i5, 9650, etc...?

2) Should I be leaning towards one socket type or L2 cache? Should I be looking for a certain TDP level?

Any and all guidance is appreciated...

[davephan]

I looked at server mainboards and Xeons before. The cost is high, features and performance are less.

I use an E6850 dual core 3.0 gig processor now. I was considering a Q9650 quad core 3.0 gig, but now am considering a i7-950 quad core 3.06 gig CPU.

The i7-950 performance is higher than the Q9650, but the cost is more. Both quads I'm considering are more performance than you may need with the SageTV computer, but I rather have a bit more processing power than not enough. CPU intensive tasks like Comskip and compressing videos is quicker with the i-7 and the 3 gig clock speed. The price drops drastically and the performance drops a bit when going down to the 2nd or 3rd from the top of the line Intel processors.

I'm still thinking about which mainboard and CPU to order. The i7 performance is attractive, but the cost is higher. Whatever I choose, I definitely want to go with a quad and keep the clock speed at 3 gigs.

Dave

[Fuzzy]

Another thing to mention is that it also really depends on WHAT you are are using the server for. Is it JUST for recording and serving up Sage? If that's the case, you could get by with any computer made in the last 5 years. Sage is VERY light on processor needs.

However, if you are looking at running comskip/showanalyser, that's a different matter, and the quad-cores might be worth it. Personally, I've gotten by fine for many years with an older AMD FX-60 dual-core, for my 5-tuner server/client/comskip machine. The only reason I'd upgrade is the FX-60 is a power hog for it's performance level... I COULD upgrade to a newer phenom II and drop the power requirements quite a bit and get a faster CPU... but it wouldn't be a big enough drop in power use to pay for itself (new CPU/MB/RAM would all be required).

[matterw]

Originally Posted by Fuzzy

Another thing to mention is that it also really depends on WHAT you are are using the server for. Is it JUST for recording and serving up Sage? If that's the case, you could get by with any computer made in the last 5 years. Sage is VERY light on processor needs.

Good point. I am using it primarily as a SageTV and Media Server (storing TBs of video, photos, music) with 4 TB for now, but I would like to go up to 20TB+. I want to make sure it was the power to allow plenty of disk throughput for future as I add more drives.

I used Comskip when it first came out, but wasn't impressed with the accuracy. Maybe I'll revisit to see where it is at. In this case, I would like to allow for CPU power for Comskip as well just in case I am pleased with the outcome.

[Fuzzy]

No matter HOW much you transfer, CPU load is always very low for network and disc access, which is all the sage server is really doing. Obviously, you can upgrade and it won't hurt, but I don't think you have any sort of limit on your current system, as a simple sage/file server, no matter HOW many drives you attach to it.

[matterw]

Originally Posted by Fuzzy

No matter HOW much you transfer, CPU load is always very low for network and disc access, which is all the sage server is really doing. Obviously, you can upgrade and it won't hurt, but I don't think you have any sort of limit on your current system, as a simple sage/file server, no matter HOW many drives you attach to it.

Currently, I have 150MB/s max disk i/o because that is the limitation of the D865PERL motherboard and the RAID card I am using. Some say the 300MB/s is hype, so not sure if this is holding me back or not. I have seen many times when the machine is recording 2-3 shows while shows are being watched and it skips or pauses.

So, based on this my thought is that the machine and RAID card can't push enough disk i/o through it. Hey, if this machine can be configured to handle more, then I am all for it -- and the savings of the $$$.

Thoughts?

[Fuzzy]

Might want to check the units there. There's a big difference between Mb/s and MB/s (8 times, actually). The PCI Bus is shared between all devices on it. This bus transfers 32-bit@33MHz. This results in a theoretical cap of 125.8MB/s. Even splitting that PCI in half (for the transfer from tuner to RAM, and then from RAM to HDD) That's about 60MB/s, or 480Mb/s. That 480Mb/s is enough for 24 20Mbps ATSC programs. Either way, you're far from saturating the PCI bus, and certainly not going to be making the processor break a sweat.

My other question for you, is it looks like you're using your 64-bit/66MHz RAID card in a 32-bit/33MHz slot? you MIGHT get better performance with a real PCI-X motherboard, but as said, I don't think your limitations are PCI related, in which case, and further storage you add would be on a separate controller, and not really take any real bandwidth from your existing drives.

[matterw]

Originally Posted by Fuzzy

This bus transfers 32-bit@33MHz. This results in a theoretical cap of 125.8MB/s. Even splitting that PCI in half (for the transfer from tuner to RAM, and then from RAM to HDD) That's about 60MB/s, or 480Mb/s. That 480Mb/s is enough for 24 20Mbps ATSC programs. Either way, you're far from saturating the PCI bus, and certainly not going to be making the processor break a sweat.

I am with you theoretically. I ran Performance Test against my server to get a Disk Mark. Based on what it retuned I am not near the 60MB/s on writes. Could it be my drives are not keeping up or are writes just going to be this way?

Here's what it returned:
1. Sequential Read: 72.5 MBytes/Sec.
2. Sequential Write: 39.6 MBytes/Sec.
3. Random Seek + RW: 2.15 MBytes/Sec.

[Fuzzy]

That's probably a combination of a limit of the drives having a slower write speed, PLUS RAID5 overhead calculating the Parity bits. Still, 60MB/s is plenty for pretty much ANY sage system.

[Savage1701]

Matt:

Stay away from my board if you are into USB Tuner Sticks. The 10 USB ports in the back are a joke and will easily crash the system "IF" you are trying to use a bunch of ATSC tuner sticks. It's too much for the hardware, and my problems ended when I bought a x1-based USB 2.0 card.

Congrats on your excellent choice of 3Ware :-). The 9500S (I assume you have the BBU) should be plenty fast BUT you can squeeze some horsepower out of it by putting it into a PCI-X slot where it will become the 64-bit 66MHz card that it was designed to be. You will essentially almost double its speed.

Also, my Asus board, under XP, is very difficult to get an aligned solid state drive to work under. If you are staying with a mechanical system drive, you will be fine. My SuperMicro boards love SSD's. If you see yourself staying with XP ( I am going to for Sage server as long as I can), and you want an aligned SSD (unaligned = slow under XP) then consider SM over Asus. I got into a running argument with the OCZ mods about alignment under XP with Asus until one of them reproduced the issue. Then their tone changed...

If I had it to do over again, I'd get a SuperMicro board. I love 'em. They don't mess around with all that overclocking junk and trying to make the board and chipset do fancy frilly things that contribute nothing to a stable SageTV server. Plus, you could easily find a model that had PCI, PCI-X (which will almost always accomadate a PCI card at the expense of some slot speed sacrifice) to let you keep using your 3Ware card, and PCI-Express for a modern video card.

I also like SM because many of their boards offer built-in or very cheap KVM over IP IPMI add-on cards. I can control a couple of my other non-Sage video servers AT THE BIOS LEVEL with this feature (Not like remote desktops that do you no good if you have a hang at the BIOS level).

For processing power, I would think even a modern single-CPU Xeon board would be great. Heck socket 775 is fine also. I'd make sure I had a good video card and a good H.264 decoder for the HD-PVR that could offload to the card. For example, Core makes a $15 decoder that specializes in offloading H.264 decodes to any CUDA-based Nvidia card (These start at about $65 and even a 9900 card series is less than $100). I tested it and my CPU usage went from 70% plus to under 5% when the Core decoder kicked in. And that was on an ancient Pentium D system.

Personally, I'd stick with SuperMicro if you know you want a Sage server and don't care about overclocking. Besides, SM answers their tech support emails. Asus, well, not so great...

Oh, and excellent choice on the Corsair PSU. Get a single-rail +12V design. Multi-rail is a joke/farce/marketting gimmick in almost all of the PSU's out there and is totally unnecessary.

[Savage1701]

Matt:

One other thing - Eboostr is viable for speeding up read-ahead caching with incredible granularity and can now safely access hidden RAM on 32-bit systems, except maybe if your video card is sharing memory. But those are few and far between outside of the laptop realm, so not really a concern.

SuperSpeed supercache can make a giant difference in speed if you allow its delayed-write cache option (Think "dirty pages"). Of course, you risk problems with any write-back caching scheme on a power failure or system hang.

Just a couple other thoughts to consider on things that can make big speed differences.

I agree also on your sustained write speed. That sounds about right for single drives or RAID 5 arrays on a card that old. I'd be more concerned about NCQ and such if you are writing multiple streams to the same or to different drives, or if one of them goes into TLER or such. You have enabled the write caching for the drives on your 3DM2 interface, correct? It's pretty much true that an ATSC signal is only about 2.5 Megabytes per second, but how that gets handled is what matters. I had to use a USB 2.0 external enclosure once with a SATA II 7200RPM hard drive and it stunk. Stutter city. Even though the throughput is miniscule, even 480 Mbp/s (really just 240 Mb/s in the real world) USB 2.0 could not arbitrate a couple ATSC tuner sticks and a single external hard drive that were moving a total of maybe 8 Megabytes per second among an external drive and a couple of internal drives for reading and writing video files. What should have been and what was were very different. I had to use a read-ahead utility to smooth the drive out until I could get it back on a SATA bus.

I would not get hung up on Xeons either in this situation. I have a SM X7SBE that can handle both a Xeon series and the 6600 core series. I just have the core duo in mine. It's plenty fast.