Massive Storage Solution(s) -- The search for a fast/expandable/HUGE storage solution

[tmiranda]

Originally Posted by stanger89

Problem with the NAS solutions is that they require another PC, in addition to the SageTV server, I see no reason to run two machines 24/7.

True, I don't like this either, but it seems to be the lesser of the evils. I can live with an extra machine:

- The NAS I have setup uses a 1.6 GHz Celeron D that draws only 40 Watts maximum (less than that when not being pushed) so the electric bill is not noticable. NASLite does not need a lot of CPU power so the performance is not effected. (I actually was running the NAS on a motherboard that had a built in 800 MHz VIA C3 and that ran just as fast!)

- I tucked the machine behind a piece of furniture in the guest room so it is not obtrusive.

- Maintaining the machine is super easy since NASLite is completely "self contained". It takes about 10 minutes to install and setup, and requires close to zero maintenance after that.

- I have 9 drives installed now and I can easily add more via eSATA if needed. I will probably just replace smaller drives with bigger drives but eSATA is still an option.

Since I have several hundred DVD's I've made no attempt to rip and store them all. Too much time, too much storage, and too much of a backup headache. I usually rip the most recent ones I am interested in and after a while delete them to make room for newer DVD's. It would be "cool" to have them all online all the time, but I'm unwilling to take on the project just to be "cool".

I have about 10,000 .mp3's that are ripped onto a single 500 GB drive. I back that up nightly to another 500 GB drive on the NAS using NASLite's "mirror" function. If the house burns down or gets destroyed in a hurricane I will probably lose both drives, but then I'll have other things to worry about besides my music collection

Like I said, we all have our needs and this works well for me.

[stanger89]

Originally Posted by mikesm

Ok, so then the real issue is that you want some level of redundancy without having the whole raid5 array spun up for access?

I'll put it this way, something like that would make it worth at least trying to get a Linux solution working. If there's nothing like that, then as you said, I might as well get a hardware RAID card and run it under Server 2003 like I do now. My server does everything I want today, but that would be a nice feature.

One thing I like about having the NAS up and running 24x7 is that I can run other things on it, which is also why personally, the appliance type solutions are less interesting to me.

I agree completely, I run other things on my server, which is why unRAID is not a viable solution for me.

After my next upgrade, I'll have a Q6600 in the core of the NAS, plus 4 GB of RAM, I plan to run a vmware instance of XP to run Sage Videotools compression and showanalyzer right on the NAS where all the content is stored. What's funny is that I just got a recent version of Norton Ghost, and one of the options it has is to build a backup snapshot in a vmware snapshot format, so you can literally backup a windows system, and reconstitute it as a virtual machine under linux... Very cool.

That is interesting.

This is not quite a NAS application, but I like the ability to run heavily I/O bound tasks right on the NAS.

Well I think the idea is if you're going to have a box that runs all the time, it might as well do as much as possible, and not require other machines running also.

Originally Posted by tmiranda

True, I don't like this either, but it seems to be the lesser of the evils. I can live with an extra machine:

- The NAS I have setup uses a 1.6 GHz Celeron D that draws only 40 Watts maximum (less than that when not being pushed) so the electric bill is not noticable. NASLite does not need a lot of CPU power so the performance is not effected. (I actually was running the NAS on a motherboard that had a built in 800 MHz VIA C3 and that ran just as fast!)

- I tucked the machine behind a piece of furniture in the guest room so it is not obtrusive.

I've already got a server and a NAS running 24/7, the last thing I want is another machine too. Another part of my whole search is I want to reduce my overall power consumption. Adding another machine will waste any gains I make with more efficient components.

- Maintaining the machine is super easy since NASLite is completely "self contained". It takes about 10 minutes to install and setup, and requires close to zero maintenance after that.

While nice, ease of setup doesn't really fall into my decision criteria.

- I have 9 drives installed now and I can easily add more via eSATA if needed. I will probably just replace smaller drives with bigger drives but eSATA is still an option.

How do you do that? Copy the data to the new drives and replace them, is is it some sort of volume where you replace and rebuild?

Since I have several hundred DVD's I've made no attempt to rip and store them all. Too much time, too much storage, and too much of a backup headache. I usually rip the most recent ones I am interested in and after a while delete them to make room for newer DVD's. It would be "cool" to have them all online all the time, but I'm unwilling to take on the project just to be "cool".

I've already got 500 online, problem is I'm having to shoehorn new ones in. And then I've got about 15 HD DVDs I want to rip, and I want to start adding BDs. I need a lot more space going forward. And with 1TB drives, it's not that big of a deal to get huge storage space.

[PGPfan]

Originally Posted by mikesm

I'm curious that if unraid supports drive spindown how much of the time the drives are actually spun down. Also, I didn't think windows liked waiting very long for a file once a request was made, but maybe that's been fixed in recent versions.

Part of the unRaid beauty is in the fact that it doesn't stripe data across multiple drives. As such, the drives are almost always powered down during use - except for the drive currently being read from. The parity drive only spins up during writes. I have twelve drives in mine, and typically only see 3 rarely 4 spun up at any time with the entire house (5 PC's) using it.

Never had trouble with any OS regarding the drive spin-up delay.

-PGPfan

[Addict]

My 2 pennies...

1) I want a single system to act as a NAS, SageTV Server, and run various other applications (backup software, commercial skipping, etc). I've already got my consolidation planned out and will be using existing hardware, just need to chunk out a few hours to make it happen. Since I got my extenders, I've moved my server from the living room (used to be client/server) to the garage with the NAS. Next step is to make it all one system.

2) I'd love to reduce power consumption on some drives by allowing them to spin down. I have a volume I use for "static" content, which is mostly videos and music. I have another volume that I use for "dynamic" content, which is TV, etc. The static content drive only gets used when I'm at home and actually watching/listening to something, so spinning it down most of the day would be great. Can't currently do this.

3) I'd love to be able to add new drives to the mix on-the-fly. I cannot do this with my current RAID controller without rebuilding everything. I'd have to move the data elsewhere, rebuild the array, and move the data back. PITA. When I'm low on space, I want to add a drive, tie it to a particular volume, and call it done.

4) I'd love to be able to have multiple arrays using different RAID configurations depending on need. Stick the OS on a RAID 1 array and various volumes in RAID 5 or RAID 10 arrays...

5) When I built my NAS, I had an old unused IDE RAID controller. Would love to upgrade to SATA but I'm not about to shell out cash for another RAID controller. A solid software model would be nice.

6) The system MUST...I repeat - MUST - be able to handle catastrophic OS failure. Meaning, I should be able to back up some kind of config file and replace it on a rebuilt OS that will be able to just take over the previous RAID config on the old system. Having OS corruption or failures destroy your RAID configuration would be stupidly ironic!

I can currently do #1, #4, and #6 today. If I could do the rest for cheap (sell my current NAS as a unit and start building new on the cheap), I'd be all over it.

AL <-- not a *nix guru

[mikesm]

Originally Posted by Addict

My 2 pennies...

1) I want a single system to act as a NAS, SageTV Server, and run various other applications (backup software, commercial skipping, etc). I've already got my consolidation planned out and will be using existing hardware, just need to chunk out a few hours to make it happen. Since I got my extenders, I've moved my server from the living room (used to be client/server) to the garage with the NAS. Next step is to make it all one system.

2) I'd love to reduce power consumption on some drives by allowing them to spin down. I have a volume I use for "static" content, which is mostly videos and music. I have another volume that I use for "dynamic" content, which is TV, etc. The static content drive only gets used when I'm at home and actually watching/listening to something, so spinning it down most of the day would be great. Can't currently do this.

3) I'd love to be able to add new drives to the mix on-the-fly. I cannot do this with my current RAID controller without rebuilding everything. I'd have to move the data elsewhere, rebuild the array, and move the data back. PITA. When I'm low on space, I want to add a drive, tie it to a particular volume, and call it done.

4) I'd love to be able to have multiple arrays using different RAID configurations depending on need. Stick the OS on a RAID 1 array and various volumes in RAID 5 or RAID 10 arrays...

5) When I built my NAS, I had an old unused IDE RAID controller. Would love to upgrade to SATA but I'm not about to shell out cash for another RAID controller. A solid software model would be nice.

6) The system MUST...I repeat - MUST - be able to handle catastrophic OS failure. Meaning, I should be able to back up some kind of config file and replace it on a rebuilt OS that will be able to just take over the previous RAID config on the old system. Having OS corruption or failures destroy your RAID configuration would be stupidly ironic!

I can currently do #1, #4, and #6 today. If I could do the rest for cheap (sell my current NAS as a unit and start building new on the cheap), I'd be all over it.

AL <-- not a *nix guru

Linux raid works well for #6. Heck, you can even take the raid disks to a completely different system with different sata controllers, and it'll recognize it and bring the volume back online (though exporting it via SAMBA will take a little extra configuration).

It also allows you do do #4. Lots of mix and match, and if you do lvm, you can even do mix and match in the same volume, not that I think that makes sense. You can even partition a disk and run the different partitions in different RAID modes, which is sometimes handy for getting RAID1 of the system disk.

Of course I am advocating software raid, which also meets #5.

As I pointed out in the other thread, you can do online capacity extension, which meets your #3 requirement too. Not being able to add a disk to an existing array, and then extending the filesystem accordingly is a huge gap in windows software raid. In my opinion, if you can't do that, you really shouldn't do raid.

Some folks have played around with spin up / spin down in linux, but I still think it's more trouble than it's worth. You should calculate how much this is really going to save you in dollars, as the disks do not burn a lot of watts when online. And the CPU, etc... all still has to be online, so in a 6 disk config maybe you save 60 watts. That's one light bulb. How much does that cost you over a year? I just can't see why it makes sense to do this. But I am open to being convinced.

#1 is where things don't work though. Windows really is currently the best platform for sage server because of driver support for tuners. And those drivers won't work well in a virtual machine either, so you kind of have to have windows as the base OS, but then you lose all the benefits of linux software raid, since windows software raid blows chunks.

So if you don't want to run two machines, then you need to deal with a hardware raid controller under windows, or deal with non-raid configuration for windows. This isn't really Sage's fault - they don't control what platforms vendors write drivers for.

[Addict]

Thanks for the thoughts...

Originally Posted by mikesm

#1 is where things don't work though. Windows really is currently the best platform for sage server because of driver support for tuners. And those drivers won't work well in a virtual machine either, so you kind of have to have windows as the base OS, but then you lose all the benefits of linux software raid, since windows software raid blows chunks.

I find this interesting. I'm surprised someone out there hasn't started developing a Windows-based software RAID controller (I admit I haven't spent much time looking). MS typically provides software that does "just enough" and 3rd parties out there build it better.

I guess the other option might be to wait for the HD Encoder devices that are hopefully coming out in the next quarter. That might eliminate some of the driver issues...assuming they're released with *nix support.

[lobosrul]

Here's my really simple answer:

I bought one of these a couple weeks back: http://www.newegg.com/Product/Produc...82E16822102004 I found it open box for $500, and added 3 750GB Samsung HDD's. Your not going to find a raid-enclosure plus a card (5 port+) for less than that.

Its only for DVD's and library files, not sage recordings. Only slight hitch I had was for some reason I couldn't add a network drive in Sage, I had to specify the share directory; that was a frustrating 2 hours. Only time will tell how it handles a drive fault, but so far so good.

I can't say how much power it draws, but its a Celeron M, supports drive spin down, and its power supply is tiny.

EDIT: Sorry I didn't see where you specified 20 TB+ (20 TB ARE YOU INSANE?!)

[mikesm]

Originally Posted by lobosrul

Here's my really simple answer:

I bought one of these a couple weeks back: http://www.newegg.com/Product/Produc...82E16822102004 I found it open box for $500, and added 3 750GB Samsung HDD's. Your not going to find a raid-enclosure plus a card (5 port+) for less than that.

Its only for DVD's and library files, not sage recordings. Only slight hitch I had was for some reason I couldn't add a network drive in Sage, I had to specify the share directory; that was a frustrating 2 hours. Only time will tell how it handles a drive fault, but so far so good.

I can't say how much power it draws, but its a Celeron M, supports drive spin down, and its power supply is tiny.

EDIT: Sorry I didn't see where you specified 20 TB+ (20 TB ARE YOU INSANE?!)

20 TB is a lot, but it's only 20 drives + parity drives. It wasn't that long ago that 20 TB filled a room.

6 ICH10R motherboard ports or 3 SI 3132's plus 5 PMP's would do it - a $100 motherboard will soon be capable of handling all that. :-)

[tmiranda]

@ stanger89 - Clearly my simplified solution is not for you. I only posted so others who do not have "uber needs" can see an alternative. It sounds to me like a Linux LVMS solution is your best choice. (Or wait a few years until 10 TB drives are available and then buy 2 or 3.)

As to replacing drives, I just move the recorded shows from the drive I am replacing to another recording drive. I try to keep the overall disc usage in the NAS below 75% so there is always room available for the move.

Edit: I think 10 GB drives have been available for a while now....

[lobosrul]

How about going with several "nearly plug and play" linux based NAS boxes? I think thats what I would do if I needed 20TB of storage.

[mikesm]

Originally Posted by lobosrul

How about going with several "nearly plug and play" linux based NAS boxes? I think thats what I would do if I needed 20TB of storage.

Volume management in that mode of use is really a pain. And you can't stripe disks across multiple boxes. If you are in that kind of league, you can probably afford to pay a local linux guru to build you a nice system.

You could put 16 disks in a compact 4U server chassis like this one: http://usa.chenbro.com/corporatesite...s.php?serno=41, and then bolt on 3U 12 disk chassis as needed for expansion ...

I wish solaris had better driver support - ZFS on a 20TB system would probably work very well.

[hemicuda]

Originally Posted by lobosrul

(20 TB ARE YOU INSANE?!)

I must have been as well. My first media server was a N*X box w/ a megaraid 438 tied to a 20bay scsi case. populated it w/ 32gb sca drives. man I wish i had the $$$$ back from that boondoggle. had a whopping 250gb after raid5 losses; 4 raid5's striped(?) into 1 logical. Dang thing had 6 power units (N+1 i think) w/ a barrelkey lock on each.

[tmiranda]

Originally Posted by hemicuda

man I wish i had the $$$$ back from that boondoggle.

In a few years we will be saying the same thing about any 20 TB server we build today. I stopped playing the "bleeding edge" game years ago (probably about the same time the kids came along), too much money, too much time, and the darn things never stay bleeding edge long enough.

[svemuri]

If any one is considering unRAID, here is my experience with it.

1. You CAN run other things on the same machine if you are inclined. the unRAID module itself is GPL and replaces the "md" driver. So you can compile it on any distro you want. The "manager" or webGUI is the proprietery part. Unfortunately, there is no real documentation or command line tool to manage unraid disks, so you are stuck with the properietery web server. PM me if you want to try this.

2. While the unraid driver itself doesn't care what file system is on each disk, the web gui uses ReiserFS and so you are stuck with it. My experience with using unRAID and ReiserFS with SageTV is not good. ReiserFS is not designed for large video files and the write/delete performance is not good. I could not get a decent playback if two HD streams (OTA) are recording. I had to abandon this setup and go to XFS to get decent performance.

3. I was kind of in the same boat as the OP. My current thinking is to just stick with vanila Linux software RAID and LVM. Actually, my current thinking is to use a RAID5 for DVD/Archive/Photos/Music etc and plain XFS file systems for recording. Like some one else said, if that drive fails, its only TV. I also thought about multiple RAID5 groups and using hdparam to spin down after inactivity period (Spin-up is automatic).

Sarat.

[SpiffyRex]

Very interesting thread. Have anyone try FlexRAID? www.flexraid.com
I really like unRAID but for static filesystems FlexRAID seems to be even more... flexible. I just want to build a RAID server and archive my DVD and movie collections so don't care about write speed. It seems like I can even make the recording drive on my Sage server part of the array and sync it over the network every night. If the recording drive fails I only loose one day worth of data.

[BobPhoenix]

Originally Posted by SpiffyRex

Very interesting thread. Have anyone try FlexRAID? www.flexraid.com
I really like unRAID but for static filesystems FlexRAID seems to be even more... flexible. I just want to build a RAID server and archive my DVD and movie collections so don't care about write speed. It seems like I can even make the recording drive on my Sage server part of the array and sync it over the network every night. If the recording drive fails I only loose one day worth of data.

Looks real interesting! I may try it next weekend myself.

[thestig]

Installed it in XP and it really works. There's also a Linux version. It's command line driven for now but they are working on a WebGUI. Oh, and you can't beat the price of $0.00 .

[stanger89]

How do you have it set up? I may like this as a "solution" to my RAID-or-not dilema when I rebuild my server...

[SpiffyRex]

Go to http://www.avsforum.com/avs-vb/showthread.php?t=1016375

Posts #12 and #13 have an step by step example for windows. Post #11 has a good/bad summary It's still a very green product but it's usable and getting better.

I'm running Ubuntu but the configuration is practically the same.
Keep in mind that it's an snapshot RAID not a dynamic. For my needs it's exactly what I was looking for. Since everything is done at the filesystem level and not at the block device level I can have remote directories (my Sage server recording drive) as part of the array. Slow to synch but who cares, I do it when I'm sleeping.

[stanger89]

Yeah, sorry meant more of a high level "what's your setup" than a low level "how'd you do it".

And yeah, I know it's a snapshot, but as you say, that's not necessarilly a bad thing. My use for it would be my DVD/movie server, and that only get's written to very rarely, so I could just kick of a snapshot update whenever I were to add a new movie. This would get me the main advantage of unRAID I wanted, that being mainly not having all the disks spinning all the time, and only spinning the required ones.