Mass Storage Questions...

[jorton]

I have been meaning to build storage system for some time and space has finally pushed me to make a decision.

I currently own 5 x 2TB WD (EARS) sitting in a nice stack all purchased before the floods at a good price.

I originally bought the 4 of the drives to use with a sansdigital 4 bay esata unit (and included highpoint rocketraid 622 pcie controller) with raid 5 but the drives didn't seem to play well even with wdidle3 applied to them. They would drop from the array (red lights on the enclosure indicator)

From what I see my only other option seems to be buy a NAS ($$$) or build some sort of home brew storage with Unraid or similar.

For a possible Unraid I have an old AMD x2 5200 on an ASUS M2NPV-VM mATX PC, 2GB DDR2 and 4 onboard Sata2 ports. I have a Silicon Image 2 port sata2 PCIe card to get me up to 6 sata2 ports total.

I'm trying to spend the least amount to get the most storage with some redundancy in case I lose a drive and possibly 20% of my recorded / ripped movies.

Are there any other options? I have a functioning WHS 2011 box (AMD x4 955 with 8GB ram on ASUS MB) with 6 sata2 ports and an addon highpoint rocketraid 640 (4 ports). Is there something I could use on the WHS2011 box that would give me a large pool like the old drive extender???

This is only for archived tv and movies so I guess speed is only an issue for playback. I won't be recording from Sagetv to these drives.

Any thoughts would be appreciated. Thanks.

[darcilicious]

Originally Posted by jorton

Is there something I could use on the WHS2011 box that would give me a large pool like the old drive extender???

Two I've heard about:

Drive Bender:
http://www.wegotserved.com/2011/10/3...ddin-whs-2011/

DrivePool
http://www.wegotserved.com/2011/02/1...e-server-2011/

[Skirge01]

There's also FlexRAID. While I haven't looked into the WHS add-ins very much (which means they might work similarly), one thing that I always liked about FlexRAID was that it didn't actually mess with your original data. Since it uses a parity solution, it actually cannot corrupt the data you're protecting. Additionally, the drives can be physically removed from the computer and accessed by any other computer.

[Dargason]

Why not use Windows 8 storage spaces? Eventually you'd have to buy it, but for now it's free.

[PiX64]

or you could say screw windows and their drive pool and go with unraid... just a thought

[jorton]

I am open to unraid, just trying to avoid having three servers in my basement

It would be nice to use the WHS 2011 box for more than nightly backups.

[panteragstk]

Crashplan for unraid works just as well.

[BobPhoenix]

Virtualize your WHS2011 and unRAID on the same box if you would rather keep your WHS backups. You could put WHS in a VirtualBox VM on unRAID. Or if your server is powerful enough ESXi with passthrough for unRAID or other NAS OS.

I actually have 3VMs:
1 -unRAID
1 -WHSv1
1 -Win7 SageTV server

[graywolf]

Yes. If you could do ESXi with your server, unRaid would work nicely.
Here is a very good thread with instruction on doing that setup/configuration.

It helped me and prior to that, had 0 experience with ESXi or any virtualization.
http://lime-technology.com/forum/ind...?topic=14695.0

[davephan]

I recommend unRAID for your video library. UnRAID is very reliable and requires no maintenance. You can easily expand the storage by adding more hard drives. WHS is very inefficient compared to unRAID (storage per number of drives). There's no easy recovery with WHS.

Dave

[jorton]

Thanks guys.

I think I'll try out unraid and see how it goes as test storage then when I get comfortable with it move my library over.

Will my old hardware be enough for decent unraid performance?

AMD x2 5200
ASUS M2NPV-VM mATX, 4 onboard Sata2 ports, Gigabit Lan (Marvel)
2GB DDR2
Silicon Image 2 port sata2 PCIe card

Also, would it be wise to have a single controller with 8 ports for all the drives to run on or is a mix of sata2 controllers (onboard and PCIe) OK?

[BobPhoenix]

Originally Posted by jorton

Thanks guys.

I think I'll try out unraid and see how it goes as test storage then when I get comfortable with it move my library over.

Will my old hardware be enough for decent unraid performance?

AMD x2 5200
ASUS M2NPV-VM mATX, 4 onboard Sata2 ports, Gigabit Lan (Marvel)
2GB DDR2
Silicon Image 2 port sata2 PCIe card

Also, would it be wise to have a single controller with 8 ports for all the drives to run on or is a mix of sata2 controllers (onboard and PCIe) OK?

For 6 drives or less using 4 onboard and 2 port would work but don't try any more then 2 ports on a 1xPCIe slot to maintain the best performance in unRAID. Going to 8 ports on your 16xPCIe slot will work IF you can put more than just a graphics card in it. If it does support more than a graphics card you might be able to use something like an IBM M1015 and an Intel RES2SV240 SAS expander to go to the unRAID 22 drive maximum. Since the PCIe slots are 1.0 that may be pushing it the extra bandwith of PCIe2.0 would help. Do stay away from the PATA ports. The Marvel lan will PROBABLY work but you may need a network card I've only used boards with Intel nics. If so I would get an Intel PCI network card to leave the 1xPCIe slot free but ONLY if it is necessary. The CPU is way more than powerful enough as long as you don't need it for plugins. I used a Celeron 430 CPU as my unRAID server until recently.

[david1234]

Originally Posted by jorton

Thanks guys.
Will my old hardware be enough for decent unraid performance?
AMD x2 5200
ASUS M2NPV-VM mATX, 4 onboard Sata2 ports, Gigabit Lan (Marvel)
2GB DDR2
Silicon Image 2 port sata2 PCIe card

That's pretty similar to what I'm running, and it just runs. The only thing I would add is to get one of the paid versions so you can use a cache drive (I haven't done this yet). Writes to the array are very slow without it, since it's always got to write the parity drive.

[jorton]

Sort of an update...

I have ordered an IBM M1015 (LSI9240-8i(???) without the Raid 5/50) with two SAS to 4 SATA cables and have the following possible hardware to use for an ESXi machine,

1) Intel Q6600 Core 2 Quad (VT-x), 8GB DDR2, Asus P5K-E Wifi/AP, with 6 onboard SATA2 ports. No video on board but I have a cheap PCIe video card that will do.

or,

2) AMD x4 955 (AMD-V), 8GB DDR2, Asus M3N78-VM with 6 SATA2 ports, onboard video.

I think I will need to get an Intel PRO/1000 GT Gigabit Ethernet PCI Network Adapter Card NIC for ESXi to work as I'm not into command line driver installs

My plan (could be crazy...) is to run ESXi 5 with a virtual WHS2011 and virtual unRaid. The IBM/LSI controller would be for unRaid.

Any thoughts on what hardware would be better or if I am going it the right direction???

Total rookie on the virutalization thing so any info tips, good install links would be appreciated.

Thanks,

Jeremy

[davephan]

I'm not sure why you would want to use WHS. WHS is very storage inefficient, only 50%, compared to only 1 parity drive with unRAID. WHS has no recovery path, other than a painful and time consuming scratch rebuild. None of your hundreds of configuration parameters or accessory programs and their configurations are captured.

If you have a hardware or software failure, how do you get the system working very quickly back to a point in time when the system was working perfectly?

Maybe there is an answer for that if you can image the host VM. It sounds like there is a lot of complexity going on there. Many things just waiting to fail! I don't know how much processing throughput is left when you virtualize verses using physical hardware for each task.

One thing you want to make sure you have covered is how fast can you recover. If your SageTV system is used everyday, by several people, it can quickly become a critical computer system to your family. You want to make sure that it is reliable and when it fails, you want to be able to recover it quickly to minimize downtime.

Dave

[graywolf]

I'm new at it also, so can only provide a little info.

Make sure vt-d capable since you'll want to be able to pass through the M1015 (or any other controller card).

M1015
Warning! These cards come with an IBM raid bios, you have to re-flash them to LSI IT-mode Bios to work.

These do not work with unRAID 4.7. You must run 5.x and newer only

Info on Flashing the M1015 here http://lime-technology.com/forum/ind...1131#msg121131

[jorton]

@ davephan - I only use WHS2011 for the consolidated backups of my 5 Win7 64 but machines, one of which is my Sagetv server and provides a quick way to restore my wiz.bin through the WHS UI. I don't use the storage options of WHS 2011 as there is no drive extender any more. Way back with WHS v1 I did try the drive extender pool thing but had nothing but issues and never trusted it so I gave up on it very quickly.

If I lose a client machine it's a quick and easy restore using a thumb drive. If the server goes I have the full server backup (new in WHS 2011) and also the clients themselves to rebuild from if needed.

@ graywolf - the eBay seller is going to flash to IT mode with the latest firmware and I am looking at ESXi 5 so it should be fine.

I'm really trying to consolidate my severs a bit as the WHS2011 box only really does anything after midnight and sits idle the rest of the day so it would be good to serve my media for the rest of the day.

[BobPhoenix]

Originally Posted by davephan

I'm not sure why you would want to use WHS. WHS is very storage inefficient, only 50%, compared to only 1 parity drive with unRAID. WHS has no recovery path, other than a painful and time consuming scratch rebuild.

I agree but he was going to have unRAID VM for mass storage. The WHS would be for backups maybe also a SageTV server.

I have two virtual unRAIDs with Win7 SageTV VMs on a SuperMicro X9SCM-F and a Tyan S5512GM2NR to hold them. I also have a WHSv1 VM on the Tyan board for backups with 4 1TB 2.5 WD notebook drives in a 4in1 cage. That is the only thing I have on it. I found that SageTV was just too sluggish when WHS was doing a backup so I split it into a Win7 SageTV server VM and a WHSv1 VM.

[BobPhoenix]

Originally Posted by jorton

Sort of an update...

I have ordered an IBM M1015 (LSI9240-8i(???) without the Raid 5/50) with two SAS to 4 SATA cables and have the following possible hardware to use for an ESXi machine,

1) Intel Q6600 Core 2 Quad (VT-x), 8GB DDR2, Asus P5K-E Wifi/AP, with 6 onboard SATA2 ports. No video on board but I have a cheap PCIe video card that will do.

or,

2) AMD x4 955 (AMD-V), 8GB DDR2, Asus M3N78-VM with 6 SATA2 ports, onboard video. Actually the whitepaper hcl lists the P5K-E as compatible so I think that combo should work fine.

I think I will need to get an Intel PRO/1000 GT Gigabit Ethernet PCI Network Adapter Card NIC for ESXi to work as I'm not into command line driver installs

My plan (could be crazy...) is to run ESXi 5 with a virtual WHS2011 and virtual unRaid. The IBM/LSI controller would be for unRaid.

Any thoughts on what hardware would be better or if I am going it the right direction???

Total rookie on the virutalization thing so any info tips, good install links would be appreciated.

Thanks,

Jeremy

I got ESXi to work just fine with my SuperMicro X7SBE but I found that the memory requirements were just too much for me. I only had 5.9GB out of 8GB left when I wanted to run passthrough. If I kept my Win7 VM as 32 bit set to 3.0GB I had enough memory to give 2GB to unRAID. I was using passthrough for both Windows and unRAID because in addition to the HBA for unRAID I had tuners for Windows for my SageTV server. If I had just network encoders I could have gone 64bit and more memory for Windows. I originally setup my X7SBE with a Q6600 to experiment but eventually put in my Q9550 for the additional speed. But I wanted my Win7 VM as 64bit so I eventually got a SuperMicro X9SCM and 16GB of memory. This gave me 13.3GB of memory for VMs.

So long story short the Q6600 would work with VT-d and passthrough if the ASUS P5K-E will. Also based on VMware communities it looks like the P5K-E should work. Actually the HCL lists the P5K-E as compatible so that combo should be fine.

[graywolf]

Originally Posted by jorton

@ graywolf - the eBay seller is going to flash to IT mode with the latest firmware and I am looking at ESXi 5 so it should be fine.

Sorry, the 5.x requirement is unRaid 5.x.
The linux kernel used in unRaid 4.7 did not have the drivers for the M1015.
So you will need to go with unRaid 5.x (most likely the latest Release Candidate would be best).
I'm on unRaid 5b11 and you could try 5b12a or the RCs.
DO NOT try 5b13 or 5b14 as they had issues with the M1015