Advice for a VM build of Sage7

[Thron]

I’m looking at building my first VM of sage v7 and want to get some input. The system is a AMD Athlon(tm) II X4 3gzh with 4GB of ram. I am thinking of using a OCZ Agility3 60GB ssd, Centos 6 KVM to do the virtualization, 2 1GB WD green drives for recording and Ceton InfiniTV 4 for HD recording. I figure having both the host (Centos 6.2 64bit) and guest (win7 64bit) on a fast SSD drive should eliminate any virtualization penalties, but I’m still concerned if the system will be able to handle 4 simultaneous recordings and 2 HD300 extenders.
I am also playing with the idea of using the green drives in a RAID 1 array. I don’t want to lose any recordings, but I also want to make sure I get the best performance possible. I know HW RAID will give me a performance hit, but will it be that much of a problem on green drives?

Any thoughts?
Thanks

[BobPhoenix]

Here is my Virtualized (ESXi) SageTV7 system.

SuperMicro X9SCM MB. X3-1230 Intel CPU (3.2Ghz I5 class Xenon). 16GB ram. 300GB VelociRaptor HD datastore. Sil3132 controller on PassThrough to eSata 5 bay encloser. Drives in enclosure are 2 3TB green, 3 1TB 7200rpm on port multiplier all with same space free and SageTV recording to most free (700GB currently free on each drive). HVR-2250 tuner on PassThrough. 3 HVR-950Q tuners on USB passthrough. 1 HDHRv1 networked (2 tuners). Win7 x64 Ultimate guest OS. WHSv1 Guest. unRAID Guest. All running and have had upto 6 things recording and one playback without a problem (for about 15 minutes anyway due to overlapping padding).

[frontlinegeek]

My only comment would be to STRONGLY suggest you use VMware ESXi over Centos.

The latest versions of ESXi are VERY close to 100% of the hardware performance. The only real issues are when you are dealing with massive database transactions and drive pools. (I use ESXi at work and swear by it bigtime).

And ESXi is free for use when installed as a standalone server (Home use)

[Thron]

I've never used VMware ESXi so I'm a little hesitant to test it on sage. Is it hard to get up to speed on?

[frontlinegeek]

Very easy frankly. There is an active DIY/whitebox crowd that even maintains an unofficial HCL for ESXi. http://www.vm-help.com//esx40i/esx40_whitebox_HCL.php

It is a bit since it has been updated but really, just grab the ISO for the latest install and toss it on and try it. The installation process is all but a total joke. The majority of the work/play is when you are setting up a VM from the Windows management console app (called vSphere Client). You can assign disk, memory and even CPU clock cycle resources to the VM. It supports both 32 and 64 bit hardware and then if you install on 64 bit hardware, you can have BOTH 32 and 64 bit guest OSes installed. You can even do single or dual core assignment to your VM.

Another BIG advantage to VMware is that it has very good support for USB and device passthrough so you can use tuners with it.

One really cute thing we figured out at work is that one of the client plugins for viewing a VM allows for its simple installation and then you can have a shortcut with credentials in it to start a console to your VM guest OS. That way you do not have to always start the full heavy mgmt client.

[graywolf]

One really cute thing we figured out at work is that one of the client plugins for viewing a VM allows for its simple installation and then you can have a shortcut with credentials in it to start a console to your VM guest OS. That way you do not have to always start the full heavy mgmt client.

Mind sharing how you do that?

I've been having to open the main VSphere Client, then from there open the Guest Console. And of course if I close the main VSphere Client, I lose the Guest Console also.

[Thron]

So are you installing ESXi on top of an existing OS or as a bare metal hypervisor? This is a bit different than the kvm hosts.

[BobPhoenix]

Originally Posted by Thron

So are you installing ESXi on top of an existing OS or as a bare metal hypervisor? This is a bit different than the kvm hosts.

ESXi is a bare metal hypervisor.

[drewg]

Originally Posted by Thron

I’m looking at building my first VM of sage v7 and want to get some input. The system is a AMD Athlon(tm) II X4 3gzh with 4GB of ram. I am thinking of using a OCZ Agility3 60GB ssd, Centos 6 KVM to do the virtualization, 2 1GB WD green drives for recording and Ceton InfiniTV 4 for HD recording.

Thoughts:

- You might be a bit light in the memory department unless you plan to devote nearly all the memory to the Windows guest.

- If you're talking about the PCIe Ceton, there is a good chance it won't work at all unless your AMD has an IOMMU. The PCI PAssthrough (IOMMU) support in Linux/KVM is better than ESXi, FWIW, but many (most?) consumer grade AMD chipsets don't support an IOMMU, and hence will not support passing a PCI card through to the guest. You should either get the USB version of the Ceton or (IMHO a better option), the HDHR-Prime. Since the Prime sits on the network, you need not worry about passing a USB or PCI device through.

- If you're going to run Linux and you're planning to use the box for anything other than SageTV (like as a general purpose NAS), then best use of an SSD is a ZFS L2ARC. I'd strongly suggest getting one more 1TB drive, and setting up ZFS with a raid-Z and L2ARC on the SSD.

I've got my main workstation / SageTV server / 7MC server setup as:

Xeon e3-1270
SuperMicro X9SCA
16GB RAM
1 x Crucial M4-CT064M4SSD2 64GB SSD
3 x Hitachi "Green" 3TB drives
Ubuntu 11.10
ZFSonLinux ppa for ZFS

I have / & swap on part of the SSD (remember to use ext4 with trim enabled), and 32GB of L2ARC on the other, with the 3 green drives setup in raid-z.

I run SageTV with 5 tuners natively in Linux, and have a KVM Win7 VM that runs 7MC and Playon. 7MC records from an HDHR to ZFS via Samba (and getting the registry hacks to record to a Samba share sure was fun!). Everything is snappy with 2 recordings in-progress, and 1 playing back on my XBox360.

Drew

[Thron]

This is giving me a lot to think about. I'm going to look at the HDHR-Prime and bypass possible PCI/USB pass through issues. drewg are you scheduling your 7MC recordings through the HD300 or from the server itself? Currently my server sits in a closet and all interaction is through the HD300s.

[drewg]

Originally Posted by Thron

drewg are you scheduling your 7MC recordings through the HD300 or from the server itself? Currently my server sits in a closet and all interaction is through the HD300s.

I schedule the recordings through the XBox360. The windows VM is essentially headless ... About the only time I rdesktop to the console is to install playon patches and stuff like that.

The win7 vm + XBOX360 is basically my "fallback" plan if my HD300 dies before there is decent support for MPEG2 on some small, low-power, embedded box that runs XBMC (like Raspberry Pi or ATV3).

Drew

[frontlinegeek]

Originally Posted by graywolf

Mind sharing how you do that?

I've been having to open the main VSphere Client, then from there open the Guest Console. And of course if I close the main VSphere Client, I lose the Guest Console also.

I'll post the info tomorrow when I am back at work so I get it right

[drewg]

Originally Posted by graywolf

Mind sharing how you do that?

I've been having to open the main VSphere Client, then from there open the Guest Console. And of course if I close the main VSphere Client, I lose the Guest Console also.

It is just vnc, wrapped in their GUI. You can connect with a normal vnc client if you tweak the .vmx and firewall rules. See
http://www.virtuallyghetto.com/2012/...to-vms-in.html

Drew

[aflat]

In addition, for any of you who run both ESXi and Workstation 7 or higher, Workstation can also connect to your ESXi hosts, and you can control most everything from Workstation. You can also connect to more then 1 ESXi host, so it's much better then 10 Vsphere clients open at the same time. Well worth the cost. My company bought in bulk, and got a price of like 50 Workstation licenses for like $79 each. It's ROI for all 50 licenses must be under a week the way i use it.

[rlvogel322]

Originally Posted by BobPhoenix

Here is my Virtualized (ESXi) SageTV7 system.

SuperMicro X9SCM MB. X3-1230 Intel CPU (3.2Ghz I5 class Xenon). 16GB ram. 300GB VelociRaptor HD datastore. Sil3132 controller on PassThrough to eSata 5 bay encloser. Drives in enclosure are 2 3TB green, 3 1TB 7200rpm on port multiplier all with same space free and SageTV recording to most free (700GB currently free on each drive). HVR-2250 tuner on PassThrough. 3 HVR-950Q tuners on USB passthrough. 1 HDHRv1 networked (2 tuners). Win7 x64 Ultimate guest OS. WHSv1 Guest. unRAID Guest. All running and have had upto 6 things recording and one playback without a problem (for about 15 minutes anyway due to overlapping padding).

This is exactly what I would like to do but with WHS2011 and I would use 2 HDHR and 1 HDHR Prime. Do you have any tips or maybe even a step by step reference on how you set it up? I'm really curious how you might be using unRAID SageTV. I'm starting to collect my components and would love to do some additional research first.

[frontlinegeek]

Originally Posted by frontlinegeek

I'll post the info tomorrow when I am back at work so I get it right

OK. Here it is. And it is not a VNC hack but a accidental discovery made by others and found out about by me.

You can use the VMware 2.0 Console Plugin that comes with VMware Server for Windows by installing the plugin onto your PC and then creating a shortcut using it and the parameters and credentials for your VMware server and the specific VM you want to connect to.

Here is an example of what the command looks like in the manually created shortcut:

"C:\Program Files\Common Files\VMware\VMware Remote Console Plug-in\vmware-vmrc.exe" -h xxx.xxx.xxx.xxx -u "username" -p "password" -m "[datastore2] Base607/Base607.vmx"

[Thron]

Thanks for all the good input, but I think I still want to keep with CentOS and KVM. I feel really comfortable with it and don’t want to experiment too much on my main recording rig. I also like the idea of using CentOS as a NAS for the rest of my network. This is how I am going to set it up.

CentOS 6.2 host and L2ARC on the SSD.
Win7 guest on KVM with a qemu image.
3 1TB WD green in RAIDz for guest image and recordings.
Bumping the ram to 12GB.
HDHR Prime CC network tuner.

I see a couple of problems that might occur.
1. The qemu image on RAIDz might be too slow. If so I can move it to the SSD drive.
2. I can’t figure out if I have IOMMU support. It looks like the AMD chip does but I can’t find anything on my mother board. That will have to be something I will figure out when I do the install. If PCI pass through works then I will attach my HVR-2250.

Is anyone using a NAS for their recording storage? Do you notice any performance hits during recording or playback?

[drewg]

Originally Posted by Thron

CentOS 6.2 host and L2ARC on the SSD.
Win7 guest on KVM with a qemu image.

I'd think twice about using CentOS. The problem is that the ZFS modules are non-KABI compliant (eg, they use symbols that could change between minor kernel versions). That means you will need to manually re-build the ZFS modules each time the kernel is updated, else you will loose access to ZFS. This was the reason I'm running Ubuntu right now, rather than CentOS -- the Ubuntu packages automatically rebuild themselves on kernel updates. This could be done on CentOS as well via DKMS, but nobody has bothered AFAIK.

Originally Posted by Thron

1. The qemu image on RAIDz might be too slow. If so I can move it to the SSD drive.

That's actually the beauty of ZFS + L2ARC -- it will cache any "hot" blocks of the file in the SSD for you.

Originally Posted by Thron

2. I can’t figure out if I have IOMMU support. It looks like the AMD chip does but I can’t find anything on my mother board. That will have to be something I will figure out when I do the install. If PCI pass through works then I will attach my HVR-2250.

If you can't find anything in the BIOS, then you probably don't. I know that AMD has IOMMU support in its recent consumer chipsets, but motherboard/BIOS vendors really suck in enabling that feature. For example, read http://www.gossamer-threads.com/lists/xen/devel/174247 and cry.. It ticks me off that lots of consumer level boards have hardware support for VT-d or AMD IOMMU, but no way to enable it due to BIOSes that focus on overclocking.

Drew

[frontlinegeek]

Drew, did you notice I posted how to do the lite console viewing for VMs on ESXi?

[drewg]

BTW, I'd suggest using this in /etc/modprobe.d/zfs:

options zfs zfs_arc_max=4294967296  zfs_vdev_cache_size=536870912

This limits ZFS ram usage to 4.5 GB, so that you've got the other 7.5GB free for your VM. Being able to limit the size of the ram cache is another advantage to ZFS.

Drew