New Server - SageTV+unRAID under ESXi 4.1

[ehfortin]

Originally Posted by stanger89

So I found another sort of stumbling block on my road to virutualized happiness. I will "shortly" need more SATA ports for just my unRAID config (5x2TB drives now), so I was looking at the 8-port Supermicro SAS card that's recommended for unRAID. However it appears it doesn't work in ESXi....

Which card are you talking about? I'm using the SuperMicro AOC-USASLP-L8i (low profile version of the AOC-USAS-L8i) under ESXi 4.1 and it is working just fine. And it is cheap (about 125$). The only problem is it seems that it is now replaced by a newer model so it not as available as it was (still find one reseller that has both form factor in Canada, may be different elsewhere in the world).

You have to order the 2x cables needed by yourself but those are pretty standard and can be found cheap just about anywhere that have SAS cabling available.

Note that even if it is called a UIO slot card only... it is actually a PCIe x4 (if I remember correctly) card with the metal bracket reversed. I've installed mine in my x16 PCIe slot and removed the metal bracket and it has been working for over 2 years now.

Hope this information help.

Have fun.

ehfortin

[stanger89]

Does it just "work" and show up as a valid place for a datastore? Can you use RDM (from the GUI) since it's SAS?

[ehfortin]

It work fine with RDM but you have to create those manually at the console. The "Raw Device Mapping" option is not usable from the GUI the way my setup is done. I'm saying it this way because I've not created any RAID group or volume into the SAS card itself so I've just let it do a "pass through" of the disks to the OS (esxi). I assume if I would have let the card take ownership of the disk, they would have been presented as SAS LUN which may have activated the RDM option in the GUI.

I've installed ESXi on an internal disk and put my main datastore on it as well. I've not tried with putting the datastore on one of the disks under the Supermicro (LSI 1068) card as I wanted raw disks on all the 8 ports for Unraid (and ZFS before it). So I've always put my OS on a dedicated disk outside of that group.

I had nothing to do in ESXi to get the card recognized. And it supports hot-plug/hot-unplugging disks which is nice.

ehfortin

[stanger89]

Interesting, I'll have to look into that one.

[chetebrown]

Hello all !

I've been reading your thread attentively and it appears that you all have the whole unraid under esxi packed down tight so I was hopping to get some insight in regards to disk temperature and spin-down.

I recently successfully completed the installation suggested here: http://lime-technology.com/forum/index.php?topic=7914 and I'm at the point where I am wondering if it is normal that unraid does not display the drive temperature or spin-down the drives.... i'm starting to think this is the normal behavior under esxi.

I see that ehfortin also encountered that issue back in early august: http://lime-technology.com/forum/index.php?topic=7261, but there was no continuity to that thread.

Ehfortin: since you have had it running for about a month would you be able to advise if this is normal behavior or if you found a way to fix that ?

Anybody else encounter this issue ?

Thanks !



CB

[stanger89]

Yeah, I think it's normal. The information and control stuff doesn't seem to make it through the virtualization/passthrough layer.

[davephan]

Originally Posted by madpoet

At full load my 7200rpm drive is at 33c and my 5400s are at 28c. The super micro does a really good job but it is a little louder than I expected.

I'm surprised that the drive temps are that low. The density is very high with 5in3's. I have a couple Thermaltake icage A2309's that are 3in3's with a 12cm fan. The drives in the 3in3's are running from 29c to 33c. I might be able to fit another four 3in3's in my full tower case to 18 drives.

Dave

[chetebrown]

Originally Posted by stanger89

Yeah, I think it's normal. The information and control stuff doesn't seem to make it through the virtualization/passthrough layer.

Since it's a vmware issue, guess i'll have to keep digging around to see if their is anything to do about this... I'm not crazy about the idea of having the drives spinning 24/7

[stanger89]

I'd bet if you use VMDirectPath to passthrough the disk controller everything would be fine.

[chetebrown]

Originally Posted by stanger89

I'd bet if you use VMDirectPath to passthrough the disk controller everything would be fine.

Definitely sounds like a viable option. The current mobo (p5ke deluxe) i'm using supports VT-D put I don't believe it supports IOMMU, at least thats the conclusion I'm drawing since the advanced configuration tab said that the host does not support this and I have yet to come across any documentation supporting this.

Still, this would result in a nice temporary work around as it's limited to only to 2 devices per VM.

Thanks for the food for thought, will be sure to update here if I come across an ingenious solution !

[stanger89]

VT-d and IOMMU are just Intel and AMD's names for the same thing, IO virtualization. Looks like Asus didn't opt to enable VT-d on that board.

I'm not sure why it would be temporary, you should be able to passthrough two, 8-port controller cards, which would give you 16 drives. Not sure what else you'd need PCI passthrough for on an unRAID VM.

[chetebrown]

Ya your right, upon looking again it doesn't support VT-D...

The passthrough of the sata controller idea is a solid idea ! But since I have to change boards and get some new sata controller cards, i'll have to postpone this until budget allows.

Since I see most people here using the pci passthrough for tuner cards and other adapters and having all their drives via RDM, has anyone had any concerns/issues about the drives constantly spinning/overheating ?

I'm going gonna go ahead and try the setup with unraid in ESXi for the week and see if it warrants getting the license for it.

[stanger89]

Originally Posted by chetebrown

Ya your right, upon looking again it doesn't support VT-D...

From Xen's wiki, it sounds like the x48 chipset supports VTd but most mfgs fail to enable it.

Since I see most people here using the pci passthrough for tuner cards and other adapters and having all their drives via RDM, has anyone had any concerns/issues about the drives constantly spinning/overheating ?

My current server had it's 8 drive array running 24x7 for probably 5+ years. I had some cooling problems early on, but once I got those sorted (I was lazy and built it before I got all my fan cages installed) no problems.

I'm going gonna go ahead and try the setup with unraid in ESXi for the week and see if it warrants getting the license for it.

FWIW, I'm not entirely sure that spindown doesn't work through RDM, I've been running the unRAID 5 beta, and it will give me the green flashing dot that normally indicates a disk is spun down. I haven't been able to determine if that's accurate or in error yet.

[chetebrown]

Originally Posted by stanger89

FWIW, I'm not entirely sure that spindown doesn't work through RDM, I've been running the unRAID 5 beta, and it will give me the green flashing dot that normally indicates a disk is spun down. I haven't been able to determine if that's accurate or in error yet.

I'm using unRAID4.5.6 and I also get the green flashing dots but I can confirm that the spin down isn't happening, simply because I've been working with the case on next to me on my desk and with the side panel open I can literally feel spinning, albeit it not a full read or write speed.

[stanger89]

Yeah, that is what it looks like. I don't suppose you tried this:
http://www.phys2virt.com/?p=26

[chetebrown]

I did actually. Unfortunately came across the same stumbling block that ehfortin did: http://lime-technology.com/forum/index.php?topic=7261.

[DigitalMan]

Originally Posted by stanger89

Yeah, I think it's normal. The information and control stuff doesn't seem to make it through the virtualization/passthrough layer.

I don't use unraid, but temperature/smart data makes it into the VM just fine for me and I can run selftests, etc.

[broconne]

Update from my ESXi system. I have been having occasional recording blips on items recorded from my HDHR - everything I recorded from my HDPVR has been perfect. I enabled support on the HDHR, waited till I got a recording issue, and then I opened a ticket with SD support. Support told me that I was getting packet loss. Support pointed me to this document which enabled me to run a stress test and replicate the condition. I seemed to only get packet loss if I was using more than a single network tuner at a time. I ran through a bunch of various tweaks - I changed network drivers from flexible to e1000, I increased my tcp/ip heap size on esxi to 130 MB, I maxed out the receive queue on both the host and the guest networks.. Still no dice, I would lose packets if I was recording more than two shows. I am unsure exactly what the problem was since I had plenty of CPU to spare when looking at esxtop.. I have three nics in this system, two on the motherboard and one pci nic - since none of my motherboard nics were compatible with ESXi. I changed my firewall nic from one of the motherboard passthroughs to an emulated e1000 and passed through the motherboard nic to Sage. At this point I seem to be able to use all 4 HDHR tuners and not get any network loss.

[harrijay]

Might want to give VMXNET 3 a try for your network adapter. right now I have 2 of them setup on my server one hooked up to my hdhr and one hooked up to the rest of the network for streaming. I have not had any issues where as when I had an E1000 hooked up I was having hiccups.

Jayson

[brainbone]

Originally Posted by stanger89

Yeah, I think it's normal. The information and control stuff doesn't seem to make it through the virtualization/passthrough layer.

Is that using physical RDM, virtual RDM or both?

I've read that physical RDMs pass "nearly all SCSI commands". I was ramping up to order a system to start testing unRaid/SageTV on ESXi 4, and was planing on using physical RDM for the unRaid drives -- but I'll need to give this more thought if they can't spin down.

Edit:
Never mind -- looks like you already answered it with this link:
http://www.phys2virt.com/?p=26

Guess I'll need to keep moving forward.