Storage questions, NAS, WOL, lots of stuff!

[Kirby]

Greetings folks. I'm in need of more space, and some answers. Here's what I have going at this point:

Sage server has 2 recording drives (820GB total, more than enough for making recordings). Storage server has 7 storage drives, with 2 more to be added (giving it 6x400 and 3x300, or 3300GB for the math impaired). Storage server recordings are moved while Sage is off, to retain program data (this point become important later).

And I'm out of space. The storage server is used for keeping shows I want long term. To expand this, what are your opinions on adding a NAS setup (ReadyNAS X6?) vs. adding external USB drives (such as the WD MyBook Essential's)? I dont run any RAID at this point, probably playing with fire huh? I guess I am willing to take a loss of data, but I also plan on swapping out drives for larger ones as the 750GB drives drop below $0.50/GB. Some concerns I have over a ReadyNAS system are that they appear to only support up to 500GB drives. Also, will they spin down with in-activity? Just seems like a decent external USB2.0 drive (actualy multiple drives) would be more cost effective, so long as I am prepared to lose data by not using RAID. Thoughts? Also, can any of the NAS systems support the large drives (750 and 960GB for instance)? Probably would only have 1-2 HD streams coming from the storage server at any time, most likely just 1.

Another thing I have been wondering about, during they day and the few hours at night that I sleep, the storage server above is just waisting electricity and lifespan of components. Is there an issue to have this system power down? Is it doing more harm than good to have to sleep/wakeup cycle every day or two? How long could it stay offline before Sage drops the guide data for the shows stored on it? And is Wake On LAN an alternative here? Or will the Sage server ping it randomly checking on shows or whatnot, making it sleep/wake several times a day? During summer months like now, I dont watch as much TV, so it could be days before I access a recording stored on one of the import directories on this server.

All my computers (except the laptop) are running on gigabit, through gigabit switches that can do jumbo frames. I've never bothered to try jumbo frames because I thought it was a no-no to do that when some devices on the LAN dont support them. Is that true? What will happen to my laptop? Is it going to choke and puke? It isnt used for Sage viewing, merely for internet browsing and sometimes I will VNC into my local machines (like the 2 above) to check things. I would like to speed up the transfers of files from the sage server to the storage server, but maybe my best option here is to setup a FTP server or something, rather than transferring through windows shares. Suggestions?

That about covers the questions I have at this time! Really good answers get rewarded with beer (where possible)!

[dvd_maniac]

One suggestion I have is to encode into a better compressible format like AVC (H.264) mpeg-4. At least your PC will be doing something at night...
I have thousands of episodes and about 1500 movies in my Sage system and I have a little over 3TB and I still have plenty of room left (About 1.2TB).

I dont run any RAID at this point, probably playing with fire huh?

Yes, "I" think you are playing with fire without raid-5.
As for a USB 2.0 device? I have the Maxtor Onetouch III 1TB external USB2\Fire800 device and I can manually set it to spin down to 3 minutes. Had it about 4-5 months now without any problems. Low heat and quiet too.

[blade]

I have no answers, but I'm curious. Do you convert your shows to a more efficient codec for long-term storage?

Of course I have to say it amazes me how much content you guys keep on hand. I rarely keep anything more than a couple of weeks.

[Kirby]

I used to do the whole encoding thing, xvid at 1280x720p and keep AC3 audio, but given prices of drives now, it seems less beneficial.

I would do h264 if it was natively supported in Sage. I wish it was! Even if I had to crank out 24-48 hours on a show to do 1080i H264 I would. But its all about playback options.

My WD external drive spins down on its own, and does very well so far, I've had 2 HD streams recording to it, and 2 different streams playing from it at the same time.

[dvd_maniac]

For me it started when I looked at my DVD rack and tried to go through it one day to find a movie to watch. Spent almost 15 minutes cleaning it up.
I had to put them all in those Slim cases to fit them all in one place so no labels showed. More than once I had to re-alphabetize them due to kids which took a few hours. More than a few times I couldn't find the DVD I was looking for. More then a few times the disc was too scratched to play. So after all this I decided to build my server and back them up. The cost of backing them up to DVD-r was greater then the extra hard drives and it is WAY more conveinent...
As for TV shows:
I get nostalgic so I have recorded a few of my older favorites and me and my wife do not like to watch shows every week but like to watch a whole season of something in like a week or two during the cold months. We hate suspense.

[Kirby]

Originally Posted by dvd_maniac

I get nostalgic so I have recorded a few of my older favorites and me and my wife do not like to watch shows every week but like to watch a whole season of something in like a week or two during the cold months. We hate suspense.

Like watching a whole season of 24 non stop?

[dvd_maniac]

Originally Posted by Kirby

I used to do the whole encoding thing, xvid at 1280x720p and keep AC3 audio, but given prices of drives now, it seems less beneficial.
I would do h264 if it was natively supported in Sage. I wish it was! Even if I had to crank out 24-48 hours on a show to do 1080i H264 I would. But its all about playback options.

http://forums.freytechnologies.com/f...t=avc+playback

About halfway down Ryokurin describes a way for Sage to play H.264 files in Sage. I play them this way now myself.

[Kirby]

Originally Posted by dvd_maniac

About halfway down Ryokurin describes a way for Sage to play H.264 files in Sage. I play them this way now myself.

I'll try that again. I didnt have much luck at the time I did try it, but I was testing with live H264 streams from Dish.

[dvd_maniac]

Originally Posted by Kirby

Like watching a whole season of 24 non stop?

Yes, as a matter of fact we did watch Season 1 that way.
We had a nasty snowstorm a few years back (almost 3 feet) and decided to start watching something new and we just COULD NOT stop watching it.
I vote season 1 of 24 as a top five TV season of all time...

[blade]

Originally Posted by dvd_maniac

Yes, as a matter of fact we did watch Season 1 that way.
We had a nasty snowstorm a few years back (almost 3 feet) and decided to start watching something new and we just COULD NOT stop watching it.
I vote season 1 of 24 as a top five TV season of all time...

I did the exact same thing this past winter and I agree on it being one of the best ever.

[Kirby]

Ok, nothing I do makes the H264 stuff play. I've had enough for tonight. 3 different uninstall/install sessions with reboots inbetween, and none of my sample H264 files will play. I dont know if its because they are in a TS container or not. But the last test I tried was to drag a recording from StarzHD into graphedit, and it only is rendering the audio, so something is screwed up.

[PGPfan]

Hi Kirby,

You might want to at least look at unRaid which seems to have all of your desired features, in addition to power management/drive spindown and the ability to 'grow' your array as you buy more storage (you can mix and match sizes too).

If you have any questions, feel free to ask. I have one, and have been very please with it so far (and cheap too )

-PGPfan

[Lucas]

PGPfan, the unRAID system looks interesting.Especially the RAID 4 ability is very flexible.

I just built a very-very similar system using the NASLite v2 software from www.serverelements.com.

Using a 8x300Gb ATA drives connected to a 3ware 7500-8 card I get a RAID 5 2TB array.

The total cost of the box was:
$100 for mobo, CPU and memory
$150 for Coolermarster Stacker case
$50 for PSU
$60 for 3x Lian-Li 2to3 Disk Drive Bays
$150 for 3ware RAID
$29 for NasLitev2
$800 for 8 300Gb Drives
Total $1340
And this is over here in Europe where prices are on average 10-20% higher(at least for drives).

I started out with an old PII PC with no cost and added 3x300Gb drives with no raid. That was also effective but no RAID and not much space for drives and ventilation.

I store my DVD collection, music, pictures and various OTA recordings that I want to keep.

[sunray]

Originally Posted by Kirby

Greetings folks. I'm in need of more space, and some answers. Here's what I have going at this point:


To expand this, what are your opinions on adding a NAS setup (ReadyNAS X6?) vs. adding external USB drives (such as the WD MyBook Essential's)?

The problem with external drives whether USB or Firewire is cooling. Most external cases nowadays dont have fans and/or the ones that do have fans use 40 mm fans that dont do a good job of cooling and are noisy. That's one of the reasons that no hard drive company, not Seagate, not Hitachi, Western Digital or Maxtor offers more than a 1 year warranty on their consumer level external drives. Even though when sold separately Seagate drives have a 5 year warranty, Hitachi drives have a 3 year warranty and some Western Digital drives have a 3 or 5 year warranty.

External drives are really meant to be used for backups which means they are off the majority of the time. Running them 24/7 is not a good idea. If you search carefully, you can find external hard drive cases with 80 mm fans. The fan blows on the drive from the side which allows a larger fan to be used instead of sucking air over the drive from back which limits the size of the fan to 40 mm or so.

Also, if you want to use external drives Firewire is better than USB2. USB2 is theoretically faster than Firewire but in practice tends to be slower and puts a greater load on the CPU. And the new Firewire 800 standard is faster both in theory and practice. The downside is that Firewire cases are more expensive than USB2. You should also consider external drive cases using the new ESATA (External SATA) standard. It's a variation of the SATA standard that allows you to run SATA drives externally with special cabling and connectors. Its the fastest external drive standard of all. And there are adaptors available to use the new ESATA cabling and connectors with regular SATA equipment.

[Kirby]

OK, so it looks like if I want a large drive array I need to just make my own, similar the the unRaid system. I have enough components for a PC, buy a big case like the Cooler Master case they show, or other similar, and I could even go about builing my own linux install onto a USB flash drive like they do it.

But things like ReadyNAS just seem to limited, only 4 drives max. I want to be able to put in say up to 12-15 drives, and be able to swap them out for larger and larger drives as they come out and as space requirements dictate.

I am aware of the firewire speed vs. USB 2 speed, and that external drives are hotter, but surprisingly the WD My Book 500GB drive I have runs rather cool, even when its reading/writing 4 HD streams at a time. A fan would definitely help, but for recording and playback, 4 HD streams over USB 2.0 works great, whereas firewire (400 or 800), or eSATA is going to require a PCI card of which I dont have an empty slot on my sage server. Now if I can find a PCI-e X1 version of a firewire card, that would be a possibility.

[stanger89]

The ReadyNAS supports 750GB drives, but last I checked, only supported 2TB (base 2) volumes, so you lose about 50GB of space (since 750GB is base 10).

[Kirby]

At this point, my best bet seems to be to upgrade my existing storage server. Its housed in a Thermaltake Armor (see here) case, and I loaded the front bays up with the Thermaltake 3 HDD tray/fan units. This gave the system a 13 HDD capacity, but if I were to get two of the Lian Li 4 in 3 trays and two Lian Li 3 in 2 trays I could up my total drive capacity to 18, as there are 3 HDD in the upper rear of the case, and one more (OS drive) at the very top front.

Bundle all that with NASLite v2 USB, already running a 700watt PSU, and get few 4 or 8 port SATA controllers (most of my existing drives are PATA). Does this NASLite v2 allow you to increase capacity later on, or does the whole array have to be rebuilt?

I'm still concerned about having up to 18 drives spinning up at once, and I havent seen spin up delays on hard drives except for SCSI. Does it exist on any SATA controllers out there? Whats the effect on a RAID 5 array if some of the drives in the array start up before others?

Also, just to keep track, looks like I owe 5 beers so far

[MeInMaui]

Originally Posted by Kirby

I'm still concerned about having up to 18 drives spinning up at once, and I havent seen spin up delays on hard drives except for SCSI. Does it exist on any SATA controllers out there?

I'm not sure if this would be useful, but you could get some of these so that maybe half of your drives could be delayed by 7 seconds on power up. The units are made for adding a second drive to a TiVo because the power supply isn't designed for the current draw of two drives spinning up simultaneously. I don't know if there would be any issues with delaying the spinup of some of your drives in this manner, but I thought I'd put it out there as a possibility.

Aloha,
Mike

[Kirby]

Thanks I will look into it. But I am thinking I might just let the drives run, rather than spin down. I've always been the type that prefers to keep the drive motor going, rather than having multiple start/stops every day. I wonder which is more beneficial to the life of the drive, or if it even matters. Given the way drives are increasing in size, I highly doubt any drive I have now, or soon to buy, will be in service 2-3 years from now.

[Lucas]

Originally Posted by Kirby

At this point, my best bet seems to be to upgrade my existing storage server. Its housed in a Thermaltake Armor (see here) case, and I loaded the front bays up with the Thermaltake 3 HDD tray/fan units. This gave the system a 13 HDD capacity, but if I were to get two of the Lian Li 4 in 3 trays and two Lian Li 3 in 2 trays I could up my total drive capacity to 18, as there are 3 HDD in the upper rear of the case, and one more (OS drive) at the very top front.

Bundle all that with NASLite v2 USB, already running a 700watt PSU, and get few 4 or 8 port SATA controllers (most of my existing drives are PATA). Does this NASLite v2 allow you to increase capacity later on, or does the whole array have to be rebuilt?

I'm still concerned about having up to 18 drives spinning up at once, and I havent seen spin up delays on hard drives except for SCSI. Does it exist on any SATA controllers out there? Whats the effect on a RAID 5 array if some of the drives in the array start up before others?

Also, just to keep track, looks like I owe 5 beers so far

This is what I did and I am at last happy with the storage I got.

The so-called online capacity expansion depends on the whether the RAID controller supports it or not.