How much storage space is enough?

[paulbeers]

I am in the process of setting up a new Sage Server. My old one is fine, but I am running out of storage in my RAID 5 array and the current case does not have any more room. Time to upgrade!

My current DVD storage array uses a Dell Perc 5i Raid card supporting 4 x 1.5TB Samsung drives for a total of 4.5TB (4.1 usable). My new server is going from a case that holds 9 hard drives to one that can house 22 hard drives. My current plan is to add 4 more 1.5TB drives to my RAID 5 array with a hot spare (my Perc 5i does not do RAID 6). Giving me a total storage of 9TB (approx 8.2TB usable). I am also adding another Perc 5i card so that I could have another whole RAID 5 array of 8 drives when I need it.

So my question is, What does everyone have in their servers for storage? Right now I am mainly storing DVD's and find that I can get about 160-180 DVD's per TB of storage (I usually just rip the Main Movie and only the DD5.1 sound track). What about blu-ray? I only have two ripped right now, so I do not have a large enough sample to know what the average blu-ray takes for storage space. Anyone have an idea?

Eventually I plan to have 21TB of DVD/Blu-Ray storage when I add another 8 drive array of 2TB drives (with hot spare as well) which should keep me rolling for a few years....

[aflat]

Blu-ray rips vary widely I'm finding. Some clock in at 17gb, others, like avatar are about 43gb. That's with no compression, just movie, 1 subtitle track(2 if there are forced subs) and 1 audio. On average most blu-rays are about 20-25gb I would say.

[Fuzzy]

theoretically, a BluRay COULD take up to 23GB/hour... not all that likely to see that in practice though.

[Bizarroterl]

If you rip BR the answer is easy - you either don't have enough storage or you soon won't.

[davephan]

It depends on how much video you need to keep! I started with several hundred gigs and I am up to 9 TB now. The last upgrade doubled my storage from about 4 TB to 9 TB and took several months to fill up. You have to be determined to delete a lot to keep the drives from filling up. Long before you reach 9 TB, you will be at the point where most of what you are storing, you will never have time to watch. You will watch some, but only a tiny fraction of what you have stored.

I setup my drives with RAID 1 pairs, which is safer than RAID 5, but far less efficient. I thought about buying a case that could hold 20 drives, each removable from the front. What kind of case are you considering?

RAID 6 is a better solution, it's much safer than RAID 5 and much more efficient than RAID 1 drive pairs. However, the controllers that I've seen that can do RAID 6 are much more expensive. I think RAID is a better solution than WHS, since you can take images of your boot drive and recover to a known good working state in the past.

Dave

[paulbeers]

I have the Norco 4020:
http://www.newegg.com/Product/Produc...-021-_-Product
It has the 20 Hot Swap bays in the front.

Further, I would love to go with better RAID cards (especially RAID6), but being able to pick up used Dell Perc 5i's for 60 bucks for an 8 port hardware RAID controller is just too good to pass up (and yes I have extra cooling blowing on them to keep them from overheating) and the ability to run RAID 5 with a hot spare (so that if one of the drives fails it will automatically be incorporated into the array) makes it almost as safe as RAID 6.

My OS "drive" is going to be two 500GB hard drives in RAID 1. I am not cutting corners on this Media/File Server.

As you stated, it has taken me quite some time to fill even the 3.XTB of storage with DVD's (up to over 600 in archives); however, I am assuming I am going to be ripping a lot more Blu-Ray in the not too distant future so that is why I am ramping up my storage and why I posed the question of "how much storage is enough". Obviously that is a subjective question, but seeing numbers like 20-45GB per that means that I could only get 25-50 per TB of storage....

I am building this server to last me awhile. No more incremental upgrades (at least as far as case/hard drives/etc).

[paulbeers]

Originally Posted by Bizarroterl

If you rip BR the answer is easy - you either don't have enough storage or you soon won't.

Pretty much what I was thinking! I only have a TB left on my current server and that will be gone pretty quickly with Blu-Ray rips!

[Fuzzy]

Originally Posted by paulbeers

Pretty much what I was thinking! I only have a TB left on my current server and that will be gone pretty quickly with Blu-Ray rips!

I think the best way to look at it is how much it adds to the cost of the media to have offline access. Consider the average blu-ray is about 34GB for the main movie. That means you could fit 50+ discs on a 2TB drive. With and average 2TB drive costing about $120, that's a little over $2 added to the cost of each BluRay you purchase. That really isn't that bad in the grand scheme of things...

Of course, at the same time, that's quite a bit more than $0.50 added to the cost of a DVD...

[Lucas]

Originally Posted by paulbeers

Pretty much what I was thinking! I only have a TB left on my current server and that will be gone pretty quickly with Blu-Ray rips!

I was and still am in a similar situation over the last 3-4 years.

Initially, as I was "building" my library I kept adding disks, built 2 NAS servers and I reached around 5TB. About a year and a half ago I build a new sagetv server with room for 16 drives and populated it with 10 drives giving me a total of about 11TB of storage. I too have a RAID5 controller(3ware 9500S-12) and have RAID5 with hot spare. I found that it took me about a year to "fill" about 2TB of free space mainly with BRRips.

I think I have reached some sort of plateau where the movies that I want to rip permanently to my server are becoming less and less and as such I estimate that I will be adding about 2TB per year over the foreseable future. I am also periodically doing a clean-up to make sure that I don't have things that I will never watch.

[davephan]

Originally Posted by paulbeers

I have the Norco 4020:
http://www.newegg.com/Product/Produc...-021-_-Product
It has the 20 Hot Swap bays in the front.

Further, I would love to go with better RAID cards (especially RAID6), but being able to pick up used Dell Perc 5i's for 60 bucks for an 8 port hardware RAID controller is just too good to pass up (and yes I have extra cooling blowing on them to keep them from overheating) and the ability to run RAID 5 with a hot spare (so that if one of the drives fails it will automatically be incorporated into the array) makes it almost as safe as RAID 6.

My OS "drive" is going to be two 500GB hard drives in RAID 1. I am not cutting corners on this Media/File Server.

As you stated, it has taken me quite some time to fill even the 3.XTB of storage with DVD's (up to over 600 in archives); however, I am assuming I am going to be ripping a lot more Blu-Ray in the not too distant future so that is why I am ramping up my storage and why I posed the question of "how much storage is enough". Obviously that is a subjective question, but seeing numbers like 20-45GB per that means that I could only get 25-50 per TB of storage....

I am building this server to last me awhile. No more incremental upgrades (at least as far as case/hard drives/etc).

Where do you buy Dell Perc 5i's RAID cards for $60? Are more available?

How loud are the original fans? Do you have to replace them with quieter fans? I read another post about either the same enclosure or a similar one where the fans were too loud. The fans had to be replaced because the original fans sounded like a jet taking off on the tarmac. What kind of extra fans did you install and where did you install them?

I could add another six drives to my full tower case, where all the drives are mounted internally in the case. l like the design of your case where the front of the drives are visible from the front of the case.

Dave

[PLUCKYHD]

Originally Posted by Fuzzy

theoretically, a BluRay COULD take up to 23GB/hour... not all that likely to see that in practice though.

Avatar is pretty close I believe would have to check again but I remember it being a big dang rip.

But yet mostly unlikely.

When I built my 16tb I never thought I would fill that sucker up well now it is getting close to full (14tb) and I am looking at a rack enclosure that 20 bay looks nice. Of course I could swap out some of the older 1tb drives for 2tb but that would just be wasting a 1tb

[paulbeers]

Originally Posted by davephan

Where do you buy Dell Perc 5i's RAID cards for $60? Are more available?

How loud are the original fans? Do you have to replace them with quieter fans? I read another post about either the same enclosure or a similar one where the fans were too loud. The fans had to be replaced because the original fans sounded like a jet taking off on the tarmac. What kind of extra fans did you install and where did you install them?

Dave

I bought my most recent one for $60 from here:
http://www.ascendtech.us/itemdesc.as...NTDELLSASUT571

However, you will probably need the bracket to install into a "normal" case which you can purchase off ebay for about $10.

I also recommend you cool them with something or add heatsinks. I use one of these to cool the Perc 5i I have been running for about a year (and will use another to cool my newest one when I get it up and running):
http://www.newegg.com/Product/Produc...82E16835425004

I also buy the SAS to SATA cables on ebay for less than 10 bucks a piece. All said and done I have about $110 in each one (although I do not have the batteries in them, I figure I probably won't be writing a lot of data during storms and have a UPS).

As for the loudness of the case, I am not sure yet. I have read the same things about the Norco cases. I have their entry level Rack mount case and it is fairly quiet for a Rack Mount but then again it only has 4 fans and this one will have 7. My servers are in a closet in the basement so unless it is ungodly, I probably won't care. These are often times purchased by people who have never worked in a server environment and aren't prepared for the noise they can generate!

[HelenWeathers]

Originally Posted by paulbeers

So my question is, What does everyone have in their servers for storage? Right now I am mainly storing DVD's and find that I can get about 160-180 DVD's per TB of storage (I usually just rip the Main Movie and only the DD5.1 sound track). What about blu-ray? I only have two ripped right now, so I do not have a large enough sample to know what the average blu-ray takes for storage space. Anyone have an idea?....

I have about 250 BDs on my PC. They average about 25GB per movie (main movie w presentation graphics, HD audio track and AC3 audio track).

I have 8TB of movie only storage in my server, but plan to only keep my favorite 300 or so movies on the server.

[Bizarroterl]

Keep in mind that RAID5/6 arrays are not expandable friendly - basically you move all data off the array, rebuild it with more drives, and then restore the data. That could be a problem with several TB of videos.

If you're looking at a NAS only solution that is expandable look into unraid. I'm using it and I've added drives twice, added a controller, upgraded several times, replaced the CPU with a slower one, and moved everything into another chassis. All of this was done and no data had to be moved.

If you're looking at adding something to a server then look at flexraid.

Both of the above offer features that are ideal for media storage. The data is protected with a parity drive. Unlike RAID5, 2 drives failing results in losing the data on those drives only, not the entire array. You're not restricted to the drive model you initially started the array with. Heck, you could start with 1TB drives and then add 2TB or 3TB drives when you want. Even an array with a couple 300GB IDE drives, a 1TB SATA, and 3 2TB SATA drives is easily done. flexraid is free and unraid is free for the 1st 3 drives. Upgrading unraid isn't expensive. When a drive isn't being used it can be spun down to save energy and to extend the drive's life (not possible with RAID5/6). The only drawback to the media centric systems is they're slower than RAID5/6. They are plenty fast to feed several full HD streams, so that's not an issue unless you need to stream 5+ full HD streams at once.

[paulbeers]

Originally Posted by HelenWeathers

I have about 250 BDs on my PC. They average about 25GB per movie (main movie w presentation graphics, HD audio track and AC3 audio track).

I have 8TB of movie only storage in my server, but plan to only keep my favorite 300 or so movies on the server.

That's exactly the kind of data I was looking for! With my collection of DVD's, I know that they take on average right about 5GBs but had no idea what the average Blu-Ray was consuming (obviously they can hold 50GB of storage, but I assumed not ever movie was using the full space).

[paulbeers]

Originally Posted by Bizarroterl

Keep in mind that RAID5/6 arrays are not expandable friendly - basically you move all data off the array, rebuild it with more drives, and then restore the data. That could be a problem with several TB of videos.

Actually the software for the Dell Perc 5i allows you to expand the array without moving all the data first and then adding a drive. I've done it myself with no issues.

[Oats]

Look into the Perc6 cards, they used to only be $30-40 more than the Perc5 cards and can do raid6. I think the SAS>Sata cables and BBUs from the 5 work on the 6 too. I'm pretty sure you can also plug your current raid5 drives into a Perc6 and have the array recognized.

Bizarroterl: Motherboard based raid5 can't be expanded, but any decent dedicated raid card is capable of expanding and migrating.

[Bizarroterl]

To a mixed drive setup?

[ytulpan]

i have 20 TB (10*2TB) of storage, about 60% full. I don't raid anything. If a disk goes, then it goes. Less guilt about not watching most of the recorded stuff.

Also, in my long history with disk faults, I was able to retrieve most information from failed disks. This is especially true if one has some kind of SMART reporting.

On the other hand, I differentially image the SSD system disk daily and keep all the backups.

[hemicuda]

kind of OT, but has the write cycle limitation of static memory been improved enough to handle being the OS drive?