Looking to virtualize my server :)

[farscapesg1]

Right now I'm running WHS on an AMD x4 9750. I've talked the wife into upgrading the server and I want to go down the virtualization path so that I can run extra VMs for certification/software training. I already got my hands on an Intel Xeon X5640 processor and I'm planning on pairing that up with a Supermicro X8ST3-F motherboard. Where I'm running into planning issues is what to run on the system Server 2008 R2 wth Hyper-V, or VMWare VSphere (yes, I know VMWare would be a type1 hypervisor as opposed to Server 2008 R2).

Couple things...
1) I currently have two Nvidia DualTV PCI tuners that I don't see upgrading soon (most of my spare cash will be used for the motherboard/ram).
2) I want to continue using WHS, just in a VM. My plan is to run the WHS OS drive in a VM and pass hard disks for storage directly to the VM.
3) I currently convert recorded TV and movies using Mediashrink to MP4 (TV) and MKV (movies) format. Since I run this on my current WHS, I want the performance to be on-par for conversion times... if not better.
4) I'm planning on running at least 2 additional VMs full time, and another 2-3 sporatically when I need to test/play with something.

I'm thinking my best option would be to run 2008 R2, with SageTV running on that install to directly access the tuner cards. My WHS would be in a VM. I'm also thinking this will give me the best performance running Mediashrink as it will see all 8 threads on the processor. I would probably need to reserve some CPU/Memory for the WHS server to make sure that it doesn't hang during media encoding.

Alternatively I could run VSphere and give the WHS server 4 vCPU (I think VSphere will recognize it as 8 cores). In theory that would give it the same performance during encoding. My concern with that is while the CPU has 8 threads, it is really a quad-core with hyperthreading and I don't want to take the chance of locking the whole system. I'm not entirely sure what I would really gain using VSphere at home (other than being used to using it at work). I guess "technically" I could run SageTV and WHS in seperate VMs since I should be able to pass PCI hardware using VSphere to a VM.

My other issue is storing my VMs. Would I really gain anything by picking up a PERC5i or PERC6i off ebay and running RAID5 to store the VMs, as opposed to just running 1 or 2 VMs on seperate SATA disks to reduce disk IO? Yes, there is the "uptime" factor if a drive fails that RAID5 would help, but from a strick performance standpoint... is there much of a reason? Heck, even with 4 VMs running on the same hard drive, I might even be okay with just running 2 drives in RAID1 for the redundancy....

Thoughts? Opinions?

[K O]

I recently dismantled my Server 2008 R2 virtualization setup in favor of a direct WHS machine and have been much happier with performance and reliability. We run all virtual servers at the office and I'm pretty satisfied, but for some reason the needs of streaming video to virtual drives + comskip and other features made it less reliable and more problematic.

I'm not saying 'don't do it', but you might be more satisfied with setting up a separate machine to play with your VM's rather than dismantle your perfectly good machine.

[farscapesg1]

Thanks for the information Were you using virtual disks for storage or directly passing the hard drives to the VM?

Comskip (as well as encoding to H.264) within a VM was one of my concerns. That is one of the main reasons I was thinking about running Server 2008 R2 and let that handle the SageTV functions, letting the WHS VM funtion only for client backup purposes and media storage. SageTV would record to drives managed by the 2008 OS, ripping/encoding would be done on the 2008 OS and the encoded file would be moved to the WHS VM for storage. SageTV would be set up to pull imported videos/music from the WHS storage (basically static files that don't need to be transcoded/comskipped/etc.).

As for "perfectly good".. I'm trying to cut down on power usage for the "always on" system. My Phenom x4 chip is 125W and the new e5640 Xeon I have is only 80W. The Supermicro board should also pull less power than my ASUS M2N-SLI board I'm using.

Do you remember any of the configuration settings you assigned to the WHS VM? # vCPUs, memory, etc.?

[DigitalMan]

I have dual L5520's in mine and I use vSphere 4.1. Virtualization performance from the X5640 should be excellent.

I have a main SageTV 6 vm connected to 2 HDHR's, a separate VM for storage, and currently a trial SageTV 7 in another vm connected to a USB 950Q tuner. I use reservations on the SageVM, run comskip, transcoding, etc. Sometimes I run batches of separate transcoding jobs in another VM at the same time. No problems whatsoever. I'm considering getting a couple more 950Q's when I migrate to SageTV 7 after release. I use rdm for all my media volumes.

For your PCI tuners, it might be simpler to go with 2008 R2, as you indicated. Sure, you could use DirectPath with vmware easily enough but keep in mind you're limited to 2 devices per VM.

As far as running perhaps 4 vm's on a mirror, I don't think you'll have a problem doing that. You can run tests to see what I/O performance you're getting and go from there.

[farscapesg1]

Thanks for the great infomation DigitalMan

I would only need to pass 2 devices (the tuner cards) through DirectPath to the VM if I went that route. Eventually, when/if SageTV supports cablecard tuners, I would probably ditch the SD tuners and go with a HDHomeRun Prime. I know some would say not to plan on SageTV supporting Cablecard, but honestly if they don't I'm probably going to have to switch to Media Center Comcast is working on moving away from all SD transmissions unless you use a cablebox/tuner and I don't feel like paying for and housing 4 boxes connected via s-video into my system (not to mention the IR or serial tuning required)

Dual L5520's... I'm jealous I really wanted to pick up two of these X5640's, but just didn't have the funds at the time. Of course then I'd need a more expensive motherboard. Of course, since I haven't ordered the motherboard yet, I'm still on the fence of buying a dual processor board like the X8DTL-3F in hopes that I can pick up another X5640 at some point down the road...

A test SageTV VM is one of the VM's I want this for also. I'm always scared when I decide to upgrade SageTV that something is going to get hosed and affect the family's TV viewing. But I don't want to run yet another box in the house (already have 3 laptops, 2 desktops, and the current server).

Out of curiousity, how many cores and memory do you have dedicated to your "production" SageTV VM? i know each core on the X5640 should outperform a core on my older X4 9750... I'm thinking I could get similar performance encoding with two cores to what I get now using all 4. Especially since I've noticed Handbrake doesn't seem to use all 4 cores on a lot of my HD recorded video...

I hadn't thought about splitting SageTV off of the WHS box since it has worked so well for me, but if I did that I would only need to assign 1 vCPU to the virtual WHS and then I could backup my seperate SageTV VM.... hmmmm

[tonysathre]

I've actually been thinking about this myself after reading the ESXi and unRAID thread. My plans were to run Windows Server 2008 R2 with unRAID in a VM. The reason was that I want to use unRAIDs software RAID5 so that I can dynamically expand the array without having to rebuild it. I've read that Windows Server 2008 doesn't support dynamic array expansion, which seems bad when it comes to scalability. My question; which OS would be better virtualized? I read that you can run VMs on top of unRAID, but I still need a Windows OS, maybe just XP, to host SageTV.

Thanks,


Tony

[farscapesg1]

Personally, if I was creating a VM just to run SageTV I'd go with Windows XP or Windows 7. If I go that route it will probably be with Windows 7... just to get away from XP As long as it is a clean install, Windows XP or 7 runs very stable in a VM environment.

I haven't played with UnRaid much so I can't really comment on it. I tried it out, as well as FreeNas, when I first started looking at moving to a home fileserver, but decided on WHS due to being familiar with Windows as well as the ease of management. Of course, now I'm hooked on the PC backup ability which has saved me from 3 system crashes so I have a tendancy to want to stick with what works, even if their duplication setup isn't the most conservative space-wise. I will say that I've never had any issues with the DE that a lot of people complain about (slow system response, stuttering, etc.).

[stanger89]

Originally Posted by tonysathre

I've actually been thinking about this myself after reading the ESXi and unRAID thread. My plans were to run Windows Server 2008 R2 with unRAID in a VM. The reason was that I want to use unRAIDs software RAID5 so that I can dynamically expand the array without having to rebuild it. I've read that Windows Server 2008 doesn't support dynamic array expansion, which seems bad when it comes to scalability. My question; which OS would be better virtualized? I read that you can run VMs on top of unRAID, but I still need a Windows OS, maybe just XP, to host SageTV.

Thanks,


Tony

If I had to choose, I'd virtualize unRAID, because it seems pretty happy in a VM and seems to perform well.

That said, my experience is only in ESXi, and I've read others post that they had bad unRAID performance when they ran it in a VM running on Windows. So I think you'd need to try yourself.

What I have managed to accomplish, if all your SageTV stuff is happy in a VM, I'd run both Sage/Windows and unRAID in VMs on a hypervisor like ESX(i).

[tonysathre]

Originally Posted by stanger89

If I had to choose, I'd virtualize unRAID, because it seems pretty happy in a VM and seems to perform well.

That said, my experience is only in ESXi, and I've read others post that they had bad unRAID performance when they ran it in a VM running on Windows. So I think you'd need to try yourself.

What I have managed to accomplish, if all your SageTV stuff is happy in a VM, I'd run both Sage/Windows and unRAID in VMs on a hypervisor like ESX(i).

Running ESXi was my initial plan, but I don't want to spend a lot of money to upgrade to hardware that is supported by ESXi. I guess I will just have to try it for myself and see if ESXi will work with what I have.

About unRAID on Windows, or vice versa, I only have 2 clients. A HD300 and an HD100. 99 percent of the time, only the 300 will be in use. I have no tuner cards, and only watch movies, one at a time obviously. Would running unRAID in a VM even with slower performance, be able to handle that? I did my movie rips at 720p because I was short on disk space.

[stanger89]

Originally Posted by tonysathre

Running ESXi was my initial plan, but I don't want to spend a lot of money to upgrade to hardware that is supported by ESXi. I guess I will just have to try it for myself and see if ESXi will work with what I have.

That's what I did, it runs (fine) on a lot more hardware than what VMware calls compatible. All I had to buy was an Intel NIC as the onboard Realtek wasn't supported.

[farscapesg1]

Out of curiosity, what sort of resources did you dedicate to the SageTV VM? If it wasn't for needing to encode to h.264, I would guess that I could run an XP VM for SageTV with only one vCPU and probably 512MB RAM. Similar for a WHS VM (except maybe 768MB - 1GB RAM). Maybe run a seperate VM just for encoding/showanalyzer...

I really just need to purchase the MB/RAM and start playing

[stanger89]

Well my "production" server is an Athlon XP 1800 with (IIRC) 1GB, I gave the VM 2GB and I 2 CPUs, but mainly because I was experimenting and wanted to try some transcoding/comskip/etc.

[DigitalMan]

Originally Posted by farscapesg1

Dual L5520's... I'm jealous I really wanted to pick up two of these X5640's, but just didn't have the funds at the time. Of course then I'd need a more expensive motherboard. Of course, since I haven't ordered the motherboard yet, I'm still on the fence of buying a dual processor board like the X8DTL-3F in hopes that I can pick up another X5640 at some point down the road...

I usually put together all my own systems (NOT my wife's it works out better for me to get her a system with a nice warranty so she can yell at someone else). I got a good deal on this one and it worked out a lot cheaper to buy it complete. If it isn't practical to get a decent DP board now perhaps you can upgrade that later.

Originally Posted by farscapesg1

Out of curiousity, how many cores and memory do you have dedicated to your "production" SageTV VM?

The Sage vm is 2GB 64-bit Windows 7. I originally had 3 cpu's in it and there wasn't ever much stress on it. I bumped it up to 4 since there was plenty of spare CPU on the host, why not. When I upgrade to SageTV 7 I may bump it to 4GB and add a couple more Sage clients. I only have a 1 CPU reservation on it but if it ever became a problem I could bump it up. Sage recordings go to drives raw mapped into that VM and imports are on arrays raw mapped into a different vm.

I took a month or so evaluating both Hyper-V and ESXi back in 2009 when I set this up, ran a bunch of tests. I suggest you do the same and see how it performs on your hardware. There are plenty of settings to tweak if you need to have guaranteed resources in the Sage VM.

[tonysathre]

Have any of you considered XenServer or Hyper-V as alternatives to ESXi? Have any of you used it?

[stanger89]

I looked at Xen but it looked like way, way too much work to get going.

[Spectrum]

Originally Posted by tonysathre

Have any of you considered XenServer or Hyper-V as alternatives to ESXi? Have any of you used it?

I did some research about a year ago using Xen as a platform and it is a ROYAL PITA. When it works it works. When it doesn't (and it usually doesn't) your going to spend hours chasing down old mailing list posts looking for a solution that may or may not work. The documentation is pretty poor and is always way behind the current dev version. The only reason I would recomend using it is if you need to be able to modify what your hypervisor is doing. I've never used Hyper-V so no insights on that

[farscapesg1]

I'll probably try out Hyper-V just to give it a shot. After looking into Xen... it looks like it would be more work than it is worth. I'm already familiar with ESXi, the Supermicro X8ST3-F motherboard I just ordered is supported by ESXi (as well as the 4-port Intel NIC I have lying around). So in the end, Xen/Hyper-V/etc. would have to provide something substantial for me that ESXi wouldn't. At least Hyper-V gives me another "enterprise" virtualization technology to play around and learn. I just don't see Xen ever being "mainstream".

Of course, once I have the new box up, I may repurpose the old AMD system to learn something different (and compare side-by-side)

[gplasky]

Windows Hyper-V doesn't support the passthru of any hardware. ESXi will be the better choice.

Gerry

[valnar]

I've always had bad luck virtualizing anything that does real-time audio or video recording. I'd say you are asking for problems.

This is a tool that is good a testing that.
http://www.thesycon.de/deu/latency_check.shtml

I suppose if you throw enough hardware at it, you can make it work for a couple tuners. But if the Sage VM is going to do a lot of multitasking (several recordings, comskip, etc) simultaneously, it will fall flat on its face.

[farscapesg1]

Originally Posted by valnar

I've always had bad luck virtualizing anything that does real-time audio or video recording. I'd say you are asking for problems.

This is a tool that is good a testing that.
http://www.thesycon.de/deu/latency_check.shtml

I suppose if you throw enough hardware at it, you can make it work for a couple tuners. But if the Sage VM is going to do a lot of multitasking (several recordings, comskip, etc) simultaneously, it will fall flat on its face.

Thanks for that link. Once I get things set up I'll definitely be testing things out thoroughly before attempting to switch over.

Worse case scenerio is I'll end up going back to my original plan of Server 2008 w/ Hyper-V and just run SageTV under the host OS and leave the virtualization for WHS and other tasks