Show "Hoarders", how much space did you use?

[IVB]

As posted elsewhere, i'm reconnecting the comcast cord as its only $10/month more (for first 12 months) than HuluPlus+TV Passes. But after my incredible annoyance with shows expiring in my HuluPlus queue, i'm thinking about becoming a hoarder, recording every freaking thing, and never deleting. (Wife & I randomly re-watch comedies).

I'm trying to figure out how much space I'd want. Hard drives have plummeted in price, a 3TB enterprise HDD is $200. I'd RAID5 it via my RocketRaid card, but online expansion takes forever so I'd just as soon start with what I want.

In my prior "active" days I only had 1TB mentally allocated to SageTV as I'd delete shows, and rarely came close to filling it. I have another 2TB in BluRays or DVDs, so I was content with a 5x1TB raid5 array.

In this brave (old) new world of hoarding, I don't know where to even start. Would I store/record 3 shows or 30? I could easily get 3TBx6 for a 15TB array, but that just sounds stupid.

So, i'm not asking you what I should do, as I realize the answer is "it depends on how much you keep". What I'd love to know is:

For those of you who hoard TV shows, how much do you hoard, and how much space do you have? (And that dude with the 35TB array, i'm not going to your level of insanity )

[graywolf]

I have an unRaid box that I keep our shows.
Some we watch and delete, some we keep.

Most shows we end up taping the full season before we watch them, especially any New shows. If they get cancelled, we just delete them unwatched.

My wife loves the old western shows and has been on a recording spree. She wants to get every episode she can of the many shows she wants.

Also have many movies ripped.

In addition to the various Media, also use the unRaid server as backup location for all our devices.

Currently around 41TB used of 50TB.

Just noticed the last line of the OP.

Just a word of advice, once you start on the "Hoarding" path, it goes quickly.
Then the Maxim is: The more Storage you have, the more you will use and therefore will wish to obtain more.

[BobPhoenix]

Originally Posted by IVB

(And that dude with the 35TB array, i'm not going to your level of insanity )

Probably me. I have 4 servers with 15TB each of recording drives not to mention the unRAID arrays I archive to. I'm trying to reduce my storage requirements but will take a looong time to complete.

[Skirge01]

Originally Posted by graywolf

Most shows we end up taping the full season

LOL! I love that people still use that phrase. At some point, I managed to convert myself to saying "recording" without realizing it. I honestly think it was when I switched over to Sage.

Anyhow, to the OP, we are definitely a hoarding household when it comes to TV and movies. I currently have ~12TB devoted to rips and recordings out of the 16TB in the server (also used for music, photos, backups, etc). Sage says I have 1359.48 hours of video content, which is <57 days. We wound up cancelling our premium channels because we had 120 movies waiting to be watched. I decided that when that gets down to about 10, we'll either go to Netflix or bring the premiums back (probably the former).

Like others, we sometimes don't watch something we recorded if it got cancelled, but I've learned to read user reviews before doing so. I was very thankful I didn't delete Defying Gravity before watching the series. The studio and the critics made it sound terrible, but those who watched it really enjoyed it, as did we.

[IVB]

sigh. thanks. I wonder if a 6x3TB array isn't that dumb after all.

Originally Posted by BobPhoenix

Probably me. I have 4 servers with 15TB each of recording drives not to mention the unRAID arrays I archive to. I'm trying to reduce my storage requirements but will take a looong time to complete.

Holy crap. In CA, i'm already at the $0.35/KwH incremental power cost, I'd end up spending $100/month just on juice for those!

[reggie14]

Wow, you're making me feel inadequate.

I've got 6TB in my server dedicated to recordings. I'm always close to capacity. I'm going to be close to capacity no matter how much I have. 6TB seems to be enough for me. I have another hard drive in my server that I could add to my list of recording drives if need be, but I haven't bothered.

I have another 10TB in an unRAID array that has my movies and some archived TV shows. That's close to capacity as well, which is more troubling to me. I'll probably add a couple more drives to that fairly soon. I've been waiting for the first unRAID v5 maintenance release to upgrade my server.

[IVB]

so you unRAID'ers have a different box for the Sage server vs the storage server?

I just upgraded my server to a quadcore 3.5GHz with 16GB RAM last year, so whatever path I choose will need to re-use that. It is a slight hassle using the RocketRAID given the simplicity of the unRAID, but its a sunk cost. Just spent $5K upgrading the home theater (new PJ, screen, speakers, receiver), so dumping another $2K on another server would be...not politically correct

[reggie14]

Yeah, my unRAID box is separate. The only alternative would be to virtualize it.

An unRAID box would be cheap. Hard drives will be the dominating cost. Other than that, the box itself could be relatively cheap.

That being said, I have very little confidence in the unRAID developer at this point. It took him way to long to get unRAID v5 out the door, and he's been very slow to get maintenance releases out to patch bugs. Community development has died down, which is kind of a big deal due top the level at which unRAID relies on that community for core functionality (e.g., pre-clearing, UPS support, etc.)

If I was picking something now, I probably wouldn't pick unRAID. I'm not sure whether I'd pick up a QNAP or build a Windows Server.

[flavius]

I never used more than 3.5 TB with my SageTV system.

A couple years ago when I increased the number of tuners I prepared another 10 TB just in case but guess what - there's just not enough good stuff on - no matter how many tuners you have.

[matt91]

Originally Posted by Wayneb

I have 20TB with a little over 5TB free, I record whole seasons of any new show I think I might watch, if a show is canceled without an ending I delete it without watching.

My wife and I have a 2-season rule. Unless it's a new "Must See!" show, we let Sage record two seasons before we start watching it. If it has stuck around 2 full seasons, we'll give it a chance. Just no reason to start watching a show to have it cancelled out from underneath you.

[reggie14]

Originally Posted by matt91

My wife and I have a 2-season rule. Unless it's a new "Must See!" show, we let Sage record two seasons before we start watching it. If it has stuck around 2 full seasons, we'll give it a chance. Just no reason to start watching a show to have it cancelled out from underneath you.

I see your point, but at the same time you could miss some quality shows that way, too. I mean, it sucks that Firefly, Freaks and Geeks, and Undeclared only lasted half-seasons, but they're still worth watching.

[Skirge01]

You might want to take a look at FlexRAID's new tRAID solution as an alternative to unRAID. If you checked out FlexRAID in the past, that older solution is now known as RAID-F. I switched to tRAID about 3 weeks ago. The developer is excellent with helping you troubleshoot by being very active in the forums.

[reggie14]

Originally Posted by Skirge01

You might want to take a look at FlexRAID's new tRAID solution as an alternative to unRAID. If you checked out FlexRAID in the past, that older solution is now known as RAID-F. I switched to tRAID about 3 weeks ago. The developer is excellent with helping you troubleshoot by being very active in the forums.

What do you get for write performance on that? The developer, who has largely been a jerk on AVS, suggested it would have significantly higher write speeds.

[stanger89]

Originally Posted by reggie14

An unRAID box would be cheap. Hard drives will be the dominating cost. Other than that, the box itself could be relatively cheap.

Yeah, that's what I was going to say. I was in a similar position to IVB before I built my unRAID system. For a long time I had a mental block on building a second machine. Then I saw the Supermicro X7SPA builds and realized that for a cost similar to a RAID card (I already had the rest of the differential parts), and power use, I could just build my own box. So right now my unRAID is an X7SPA with a Supermicro add-in SAS/SATA card. It's been quite nice, though performance isn't that great, but it's a media server so it's not a big deal.

That being said, I have very little confidence in the unRAID developer at this point. It took him way to long to get unRAID v5 out the door, and he's been very slow to get maintenance releases out to patch bugs. Community development has died down, which is kind of a big deal due top the level at which unRAID relies on that community for core functionality (e.g., pre-clearing, UPS support, etc.)

FWIW, preclearing isn't something I've seen anywhere else. I know my ReadyNAS didn't have that. UPS support should be built in though I agree.

I guess I find your comments interesting, I've never followed unRAID closely enough to be able to make such a determination on my own (development speed/community activity). I can see V5 has been slow coming, but releases have been "consistent" if slow, so it doesn't feel abandoned at this point.

If I was picking something now, I probably wouldn't pick unRAID. I'm not sure whether I'd pick up a QNAP or build a Windows Server.

That's kind of the trouble, I've got 9 discs in my unRAID, I'm not sure I could buy a standalone NAS that large, and if I could, it would likely be enormously expensive. That's one of the reasons I went unRAID in the first place, all the standalone NASs are way too expensive for what you get (namely max size of 4 discs or so).

Originally Posted by reggie14

What do you get for write performance on that? The developer, who has largely been a jerk on AVS, suggested it would have significantly higher write speeds.

Yeah, that's what I've noticed, even on their own forums to a degree. When I tried it, way back, I asked about writing to the pool (it didn't handle that at all gracefully at the time). It felt like I was told I was an idiot for wanting to do that and to go pound sand. On top of that there's the fact that the developer has a predilection for disappearing from time to time.

[BobPhoenix]

Originally Posted by stanger89

That's kind of the trouble, I've got 9 discs in my unRAID, I'm not sure I could buy a standalone NAS that large, and if I could, it would likely be enormously expensive. That's one of the reasons I went unRAID in the first place, all the standalone NASs are way too expensive for what you get (namely max size of 4 discs or so).

That's my problem as well. I have three virtualized unRAID servers and 1 HP N40L unRAID server. The N40L has the smallest drive count - 7 drives. The 3 virtualized servers have between 13 and 19 drives each.


So if you add in the unRAID space to the recording drives I have 176TB total space - see below for updated figures - it's gone up a little since then. I'm hoping to get half of that back to use as off line backups by eliminating duplicates (using symlinks from one server to another), going through recordings and deleting shows I just will never get to again and compressing data (if I have to) to get the drive count cut in half. Then those removed drives become the offline backups. That will take me several years to complete I figure.

Update spaces:

unraid 1: 46TB 64% used
unraid 2: 32TB 50% used
unraid 3: 45TB 81% used
unraid 4: 10TB 55% used
SageTV1:  15TB 2.783TB free
SageTV2:  15TB 6.916TB free
SageTV3:  15TB 4.163TB free
SageTV4:  15TB 5.750TB free *will be getting addition 3TB recording space in next two weeks
Grand total 196TB

[Skirge01]

Originally Posted by reggie14

What do you get for write performance on that? The developer, who has largely been a jerk on AVS, suggested it would have significantly higher write speeds.

My average speed is around 55MB/s. I'll see peaks at 70+ and dips below 50, but it's generally around that 55 mark and that's with real-time parity protection.

I have to agree with how the dev comes across at times as I've personally run into it myself on his forums. But, I've had that with nearly every dev I've ever dealt with. Maybe that's just the way most developers are? I will say that I've found nearly everyone over on AVS has a stick up their arse. There's a wealth of knowledge over on AVS, but I feel like I'm dealing with a pool of doctors with a god complex. I get it, you're smart. No need to belittle me.

Originally Posted by stanger89

Yeah, that's what I've noticed, even on their own forums to a degree. When I tried it, way back, I asked about writing to the pool (it didn't handle that at all gracefully at the time). It felt like I was told I was an idiot for wanting to do that and to go pound sand. On top of that there's the fact that the developer has a predilection for disappearing from time to time.

That would've been what's now known as RAID-F. I tried it way back when, too, and dropped it for WHS' DE. The whole snapshot concept wasn't a good fit for recording to and the real-time feature he added didn't seem to be able to handle lots of changes. But, the basic RAID-F (no parity; just storage pooling) worked fine when I dropped WHSv1. So far, tRAID seems quite capable.

The dev used to have a habit of disappearing (experienced that first-hand years ago), but that seems to have been before he turned it into a business. We'll see what happens.

[vividweb]

6TB of recording space that is usually 75% full. Lots of stuff gets recorded, but never watched as it is dumped. Also have a Drobo-FS to archive to and store rips.

[stanger89]

Originally Posted by Skirge01

I have to agree with how the dev comes across at times as I've personally run into it myself on his forums. But, I've had that with nearly every dev I've ever dealt with. Maybe that's just the way most developers are?

I know Jeff isn't like that, nor any of the Sage folks. I don't know, haven't had much dealings with others. I can say I don't usually see developers trying to start a fight with their competitors though.

I will say that I've found nearly everyone over on AVS has a stick up their arse. There's a wealth of knowledge over on AVS, but I feel like I'm dealing with a pool of doctors with a god complex. I get it, you're smart. No need to belittle me.

Yeah, I don't know what happened over there, but there's definitely a few over there that I have no interest in dealing with.

It used to be a lot like here (and still is largely outside the HTPC forum). Maybe it's just all the nice, knowledgable folks that used to use HTPCs have moved on as other devices have caught up, and now we're left with just the "kids".

[IVB]

Originally Posted by stanger89

Yeah, that's what I was going to say. I was in a similar position to IVB before I built my unRAID system. For a long time I had a mental block on building a second machine. Then I saw the Supermicro X7SPA builds and realized that for a cost similar to a RAID card (I already had the rest of the differential parts), and power use, I could just build my own box. So right now my unRAID is an X7SPA with a Supermicro add-in SAS/SATA card. It's been quite nice, though performance isn't that great, but it's a media server so it's not a big deal.

Ok, that might be the clincher for me; the RocketRaid3540 is a sunk cost. Huge sunk cost, was something like $700, but this was in '08 and I had been bitten many times by HDD failure. IIRC unRAID wasn't all that back then.

its working great now (knock on wood), so I really just need to figure out how many drives to start with. It has Online Capacity Expansion, but that would require action and i'm just so damn lazy.

I still can't believe i'm back to this, but the HuluPlus show expiration really is annoying and comcast deals just for new users is cheap. That short-term teaser rate is also annoying, why they don't reward longtime users and instead encourage churn is beyond me. I could just switch between Dish, DirecTV, and Comcast every 12 months, but again, more work.

[stanger89]

I know the provider I'm on, basically if you just call them every year, they'll put you on a new "deal". Yeah, it's stupid that that's necessary but whatever.