unRAID

[wrems]

I'm thinking about building an unraid server or similar. Wondering if anyone has any advice/opinions about unraid. I'm trying to determine what the best combination of hardware is best. I don't want to spend a great deal and am trying to keep the electricity down to a minimum. Still taking in all of my options right now so any advice/direction would be greatly appreciated.

[PGPfan]

I've had my unRaid for years now. I LOVE it! The rare times I've needed support, if I couldn't find the info I needed on their forum, Tom (the developer) has always been available with assistance.

My setup uses an older Intel D865 series of board (the original recommended hardware), and currently only IDE drives. I will be upgrading to SATA as time and budget allows (just adding a SATA controller, not replacing any hardware). The great thing about the newer versions of unRaid, is that they've really opened up options for supported hardware to the point that it's not hard to find compatible stuff at all. Check out the website for the specifics.

All in all, you really can't go wrong with it! Works great with Sage - and with the latest build the only shortcoming folks had with it (poor write speeds) has been overcome. It's free to try on a 3-drive array - why not give it a shot?

-PGPfan

[reggie14]

I just put together an unRAID server last week. I haven't made up my mind about it yet. While I'm not going to go so far as to recommend that you not buy it, I have to say I haven't been terribly impressed.

First, the unRAID interface is pretty awful. I don't mean like SageTV awful- I mean like freshman-year computer science project awful. But, (kind of like Sage), the next version is suppose to work on that. I really do think the interface, and sort of the general way unRAID works, makes it a lot harder to configure it if you're not going to use it stock.

Second, the documentation is in pretty bad shape too. There's a manual that covers the basic things, which isn't too bad. There's an FAQ that covers more stuff. But, generally, the FAQ points to forum posts. From what I've seen, the forum posts often don't give directions for how to deal with the problem, but show someone (or a group of people) working through a problem. Sometimes the specific things done in the posts don't quite work because of little changes in the unRAID software. In a much smaller set of cases, it seems like some things you run into in the documentation simply aren't relevant anymore. I think the community is working on trying to improve the quality of the documentation though, but they still have a long way to go.

Third, some functionality that really ought to be built-in, like UPS support, isn't. There are ways of adding it, but I find that sort of amateurish. I think unRAID leans on their community a lot more for what I'd call core functionality than Sage does.

Fourth, some things are just kind of quirky. For instance, FTP comes enabled by default, but it doesn't work out-of-the-box without re-doing some permissions. I'm not sure why the developer would enable it if it doesn't work properly without making other changes.

Five, its a niche product. I'm sort of bitter about unRAID right now because I can't get it to work with Acronis Backup and Recovery. It's quite likely a problem with the Acronis software, and not unRAID, but the potential problem with running niche software is that other software vendors aren't going to make sure their products work with it.

Some people are concerned about write performance with unRAID. That's actually what made me nervous initially. It's actually pretty good for me. I can write to user shares at around 30MB/sec, which is better than I expected since I'm using 5400RPM drives.

On the other hand, there doesn't seem to be much else out there like unRAID. I don't get the impression that FlexRAID is any better off. WHS and RAID5 NASes are potential alternatives, but they offer fairly different features than unRAID. So, you definitely have to balance the issues above with your desired features. It certainly could be that unRAID is still your best option.

[PGPfan]

Originally Posted by reggie14

I just put together an unRAID server last week. I haven't made up my mind about it yet. While I'm not going to go so far as to recommend that you not buy it, I have to say I haven't been terribly impressed.

First, the unRAID interface is pretty awful. I don't mean like SageTV awful- I mean like freshman-year computer science project awful. But, (kind of like Sage), the next version is suppose to work on that. I really do think the interface, and sort of the general way unRAID works, makes it a lot harder to configure it if you're not going to use it stock.

Reggie, the reason it 'appears' amateur-ish is because unRaid is designed to be a "simple media server" - nothing else. It's user interface is designed to support this required feature set - nothing else. Over the time, users have requested the ability to 'extend' the feature set and that is the ultimate goal - but as Tom (the developer) has stated many times - a users data integrity is first and foremost when writing storage software, this single philosophy is why adding features has been so slow. Given the two, I'll keep my data anytime.

Originally Posted by reggie14

Second, the documentation is in pretty bad shape too. There's a manual that covers the basic things, which isn't too bad. There's an FAQ that covers more stuff. But, generally, the FAQ points to forum posts. From what I've seen, the forum posts often don't give directions for how to deal with the problem, but show someone (or a group of people) working through a problem. Sometimes the specific things done in the posts don't quite work because of little changes in the unRAID software. In a much smaller set of cases, it seems like some things you run into in the documentation simply aren't relevant anymore. I think the community is working on trying to improve the quality of the documentation though, but they still have a long way to go.

True, but again it's all (or at least the huge majority) related to adding features. Remember, it's designed to be a media server with true parity protection - not a full blown, full featured Linux distro.

Originally Posted by reggie14

Third, some functionality that really ought to be built-in, like UPS support, isn't. There are ways of adding it, but I find that sort of amateurish. I think unRAID leans on their community a lot more for what I'd call core functionality than Sage does.

See the reply above.

Originally Posted by reggie14

Fourth, some things are just kind of quirky. For instance, FTP comes enabled by default, but it doesn't work out-of-the-box without re-doing some permissions. I'm not sure why the developer would enable it if it doesn't work properly without making other changes.

In the beginning, there was no permissions/security. The reason it was added is that many users asked for it so that they'd be comfortable using unRaid in the soho space - again an area it wasn't originally designed for. The developer has responded to customer needs by adding support, but again the focus is on maintaining data intergrity for the user base.

Originally Posted by reggie14

Five, its a niche product. I'm sort of bitter about unRAID right now because I can't get it to work with Acronis Backup and Recovery. It's quite likely a problem with the Acronis software, and not unRAID, but the potential problem with running niche software is that other software vendors aren't going to make sure their products work with it.

Very true, it is indeed a niche product - in fact, there isn't anything, anywhere (to my knowledge) that implements RAID in the same way.

How are you wanting to use Acronis with unRaid? Are you trying to use unRaid as the Acronis data store? Or use Acronis to backup unRaid?

Originally Posted by reggie14

Some people are concerned about write performance with unRAID. That's actually what made me nervous initially. It's actually pretty good for me. I can write to user shares at around 30MB/sec, which is better than I expected since I'm using 5400RPM drives.

This is actually one of the newest improvements (as of 4.5.3), it's about double the speed of the previous versions (when not using the cache drive).

Originally Posted by reggie14

On the other hand, there doesn't seem to be much else out there like unRAID. I don't get the impression that FlexRAID is any better off. WHS and RAID5 NASes are potential alternatives, but they offer fairly different features than unRAID. So, you definitely have to balance the issues above with your desired features. It certainly could be that unRAID is still your best option.

FlexRAID is an attempt to mimick what unRaid does using Java (if memory serves me correctly) but it's no where near as simple to use, and doesn't provide 'live' parity protection like unRaid does.

Imho, WHS isn't even comparable (and I use a WHS server for Sage) do to the spacially inefficient 'mirroring' that it uses for file protection. Raid5 is out (for me) because of the need for identical drives, and the inability to grow an array as you add drives.

-PGPfan

[reggie14]

Originally Posted by PGPfan

Reggie, the reason it 'appears' amateur-ish is because unRaid is designed to be a "simple media server" - nothing else. It's user interface is designed to support this required feature set - nothing else. Over the time, users have requested the ability to 'extend' the feature set and that is the ultimate goal - but as Tom (the developer) has stated many times - a users data integrity is first and foremost when writing storage software, this single philosophy is why adding features has been so slow. Given the two, I'll keep my data anytime.

I'm not completely sure I buy that characterization. The fact is, Lime-Tech is a one-man company, and it kind of looks like he does unRAID on the side. So, he's very limited in how much work he can do on it. And, at least up until this point, the interface has been a low priority.

I somewhat agree the admin interface does the basic stuff fine. But, I think a lot of people are going to want to move past the really basic stuff that's easily accessible in stock unRAID, and I think the admin interface really gets in the way when you get to that point.

True, but again it's all (or at least the huge majority) related to adding features. Remember, it's designed to be a media server with true parity protection - not a full blown, full featured Linux distro.

I'm not sure its a huge majority. The FAQ basically just covers core functionality. And the FAQ responses have the problems I described above.

Certainly a lot of discussions on the forums relate to user-added features. But, the fact that that's needed I think further highlights missing features from the core product. For a file server that is so sensitive to improper shutdowns, you'd really expect UPS support included out the box. The unRAID menus are pretty spare, so it must be extremely common for people to use unMenu to get additional features, like being able to see SMART data, that you'd ordinarily expect to find out-of-the-box.

I actually think unRAID embraces their reliance on the user community. I thought unRAID 5 was suppose to make it easier to add and manage user-created addons.

In any event, you must agree the documentation really is in poor shape for a commercial product, don't you? It certainly doesn't compare well to documentation for major products, and doesn't even compare well to SageTV's user manual.

In the beginning, there was no permissions/security. The reason it was added is that many users asked for it so that they'd be comfortable using unRaid in the soho space - again an area it wasn't originally designed for. The developer has responded to customer needs by adding support, but again the focus is on maintaining data intergrity for the user base.

Right. Though frankly, I think it was pretty short sighted to develop a file server without access control mechanisms. And it sounds like unRAID has really paid for those early design decisions. But, I think there might be big changes in version 5.

However, what I'm talking about isn't exactly related to that. FTP can be made it work, and its actually not too bad. As far as I can tell, there doesn't seem to be a good reason that the settings you have to change to get FTP working weren't the default. Alternatively, I don't see why FTP was turned off by default. That just seems sloppy to me.

How are you wanting to use Acronis with unRaid? Are you trying to use unRaid as the Acronis data store? Or use Acronis to backup unRaid?

I'm trying to use unRAID as a data store for backups of my workstation, laptop and Sage server. I have a thread over on the Lime-Tech forums where I'm trying to work through this problem. I've also spent about 5 hours on the phone with Acronis tech support, and have gotten the case escalated through 2 tiers of support, and its now been passed off to the developers. And I just sent some stuff to Lime-Tech last night to see if he knows what's going on. While it mostly looks like an Acronis problem, I've found some packets coming from the unRAID that look suspicious, but I have no idea if those are the source of the problem since I don't know the SMB protocol that well.

Imho, WHS isn't even comparable (and I use a WHS server for Sage) do to the spacially inefficient 'mirroring' that it uses for file protection. Raid5 is out (for me) because of the need for identical drives, and the inability to grow an array as you add drives.

I agree that the features of unRAID are very compelling. I just wish the implementation were a bit more polished.

[stanger89]

Originally Posted by PGPfan

Raid5 is out (for me) because of the need for identical drives, and the inability to grow an array as you add drives.

Small correction, you absolutely can add drives to a RAID-5 array.

[brainbone]

I agree with all of reggie's comments -- including that there is nothing else like unRaid at this time.

In its current state, you'll likely need to use some of the unofficial addons to get unRaid to do what you need, be it for notifications, UPS support, or something else. It is frustrating at times, but at least you have the option of doing it -- much like SageTV.

While many pin their hopes on unRaid 5 for adding native support of <insert feature here>, it looks more like unRaid 5 will be focused on providing an API for making third party addons more seamless and easy to install.

Unraid is not very hardware demanding (feel free to choose a slower, low power CPU), but can have some compatibility problems (NICs, sata controllers, etc.) There is no official hardware compatibility list, so the best thing to do is select a motherboard and post it in the unraid forum to see if anyone else has had luck with it.

[wrems]

Good to know. I have surmised as much about the specs needed. I'm looking at something Atom based, probably dual core atom... Low power and descent price. I'm stuck finding a motherboard/embedded atom cpu that I like. Tricky finding a nice balance of Sata ports and PCI-e slots for future expansion, a lot of the boards I find have regular PCI slot, and only 1 of them. Still reading lots on their forum too to put all this into perspective. Thanks for the input. Been tossing around the idea of Freenas as well, who knows...

[brainbone]

Semprons don't consume that much more than Atoms (especially if you underclock/undervolt), and may open up more MB options.

[reggie14]

I went with this CPU/motherboard pair, and it seems to work well with unRAID:

GIGABYTE GA-MA785GT-UD3H

AMD Sempron 140 Sargas 2.7GHz 45w

I kind of wonder how much overall power consumption differs between a system with a moderately low-power Sempron CPU, and a system based on an Atom. I think you'll have a hard time finding an Atom board that have very many SATA ports and/or PCI/PCIe slots.

[wrems]

Reggie, did you ever put a killawatt to your system listed? How many hard drives? I'm going to start with 3 1.5TB drives.

[blade]

So is anyone using unraid as their recording drives or just for long term storage? The last time I looked at unraid I think it was suggest that it not be used for recordings due to the poor write performance.

[perfessor101]

Hey Reggie14, (sorry to Hijack wrems)
I'm using Acronis True Image 2010 under windows 7 and windows XPSp3 and writing to a user share on UnRaid 4.5.3 Pro without user security and it has been working fine so far.

I put UnRaid into suspend mode when I go to work ... or sleep ... and use a wake on lan program to wake it up with a magic packet about 5 minutes before my backup process begins on each system.
You may not need the wake on lan command but this batch file helped with Acronis working more smoothly in my setup.

wolcmd.exe 001122334455 192.168.0.33 255.255.255.0
sleep 15
dir \\storage\backups\sagetvserver
now Wakeup >>\\storage\backups\sagetvserver\wakeup.txt
sleep 15
dir \\storage\backups\m3a-win7-rtm
sleep 15

I wake it ... do a directory or the backups location ... and just to make sure the drives spin up as I am caching directories I append the date and time to a file in the backup directory. I sleep for a total of 45 seconds in the batch file because I think that's the maximum time a 'dirty' block will reside in memory before being written. (just making sure that disk is spun up before Acronis starts)


Wrems ... the last time I put a killawatt to my UnRaid it maxed at 170 Watts and was running around 110 after bootup.
The motherboard I have is an Asus P5Q-E with and intel E5200 processor and 8 GB ddr2-800 memory
I have 2x 750GB Hitachi, 2x Seagate 1TB, 4x WD 1TB, and 4x 1.5TB WD drives currently with a 500GB Seagate cache disk
All in a Coolermaster case with the Coolermaster Stacker 4 in 3 drive cages.

There has been some consternation with Gigabyte motherboards as of late as some of the recent bioses use an HPA (bios backed up to hard disk) feature and have a nasty habit of backing themselves up onto a drive and possibly corrupting data on linux file systems.


blade ...
currently I'm just using my unRaid for longterm storage ... when the off TV season hits with the improvements to unRaid over the last year I will try recording to it directly.

Hope some of this may help,
Bobby

[reggie14]

Originally Posted by wrems

Reggie, did you ever put a killawatt to your system listed? How many hard drives? I'm going to start with 3 1.5TB drives.

I don't have one. Sorry. But, it's probably just a little bit less than what perfessor101 has, since I'm using a lower power CPU and on-board video.

Originally Posted by blade

So is anyone using unraid as their recording drives or just for long term storage? The last time I looked at unraid I think it was suggest that it not be used for recordings due to the poor write performance.

I'd be pretty worried trying to use it as a recording drive. Basically, when unRAID does a write operation, it has to do 4 disk operations (2 reads and 2 writes). Honestly, it would probably be fine for 1, or even 2 simultaneous recordings, but you'd hit a wall at some point.

Originally Posted by perfessor101

Hey Reggie14, (sorry to Hijack wrems)
I'm using Acronis True Image 2010 under windows 7 and windows XPSp3 and writing to a user share on UnRaid 4.5.3 Pro without user security and it has been working fine so far.

I've tried the True Image 2010 trial on my Windows 7 box, and it works fine. It's just Acronis Backup and Recovery Workstation that doesn't work. And actually, its only Backup and Recovery under Windows 7 (in my case, Win7 x64 Pro). It seems to work fine under WinXP. But, most of my boxes are Win7 now.

Originally Posted by perfessor101

All in a Coolermaster case with the Coolermaster Stacker 4 in 3 drive cages.

Good choice. I have the same case and cages (presumably you have the CM590).

Originally Posted by perfessor101

There has been some consternation with Gigabyte motherboards as of late as some of the recent bioses use an HPA (bios backed up to hard disk) feature and have a nasty habit of backing themselves up onto a drive and possibly corrupting data on linux file systems.

I have a Gigabyte board and found that at least on my motherboard HPA default to off (actually, it had a different name). That seems to be the norm now, based on other people's comments on the unRAID forum.

[Fuzzy]

Originally Posted by reggie14

I'd be pretty worried trying to use it as a recording drive. Basically, when unRAID does a write operation, it has to do 4 disk operations (2 reads and 2 writes). Honestly, it would probably be fine for 1, or even 2 simultaneous recordings, but you'd hit a wall at some point.

I think this is my biggest concern over unRAID. They make a big point about how it isn't striped, but that same lack of striping is what causes the significant performance penalties. RAID5+Spare, or RAID6 both seem SOOO much more suited, even in software form, to unRAID for a high performance, high capacity redundant array.

Really disappointed that windows7 lost the RAID5 feature of previous versions... I'm now looking at VBoxing a linux system on my win7 sage server, just to run a software RAID... sad really...

[reggie14]

Originally Posted by Fuzzy

I think this is my biggest concern over unRAID. They make a big point about how it isn't striped, but that same lack of striping is what causes the significant performance penalties. RAID5+Spare, or RAID6 both seem SOOO much more suited, even in software form, to unRAID for a high performance, high capacity redundant array.

Sure, unRAID isn't really well suited as a high-performance file server. But the lack of striping is also a major feature. If you do have a multi-drive failure, you don't lose everything. But, from a practical perspective, it's probably the case that if you lose more than a couple drives that you might as well lost everything. So, RAID6 is maybe roughly as good.

There's a power savings issue too, though. When you write to unRAID you spin up 2 drives: the data drive you're writing to and parity drive. If you're reading then only 1 drive spins up. With RAID all of them have to spin up.

I didn't really have a need for uber-fast writes, but I did want moderately fast writes. I'm quite happy with my ~30MB/sec write speeds.

[Fuzzy]

Personally, for a sage server, I don't like drive-spin down anyways. The delay when choosing a file, and then waiting for the spin-up time, is annoying at best. I just leave drives up full time. There really isn't THAT much power use from a hard drive.

[perfessor101]

I've got a cooler master stacker 810 case with three 4in3 cages and a three in two cage.

I think when I did the power testing it only had a couple drives and no extra cards in it (other than video) and 2 GB ram ...

Now I have 8GB ram and 2x RC-218 four port sata cards.

Currently it maxes at 335 Watts on boot and on wake ...
During a parity check it floats between 220 and 230 watts
With all drives spun up it floats between 180 and 190 Watts
With all drives spun down it floats between 110 to 120 Watts
When sleeping it draws 4 watts

I can add two more drives and then I'll be needing a new case ...
I might get the Norco rpc-4220(4?) case along with two of the Supermicro AOC-SASLP-MV8 cards.

Looking at the power draw ... good thing my unRaid sleeps while I am and while I'm at work ...

Bobby

[Fuzzy]

Sure, full system sleep is important, and DOES save quite a bit. Individual drive spin-up and spin down really isn't that significant. (your 13 drives equated to a 70W difference, meaing only 5.3W per drive. Hardly worth enduring the spin-up wait.

[perfessor101]

with over 6,000 recordings ... you don't notice the spinups either ...

I've tried raid cards and other software raids before ... unRaid works ... and has an extremely helpful community ... quite a lot like here.

I tried just about every linux NAS (and windows software raid-5 mode) as well as learning Gentoo inside and out to roll my own software raids

I feel much more secure with unRaids solution on my budget ...

I could go with 3ware and spend $825 for a 12 port raid card ... or I could start with an old motherboard and three hard disks and grow it as I can afford.

For about three years I did everything I could afford to do to avoid getting unRaid ... but knew it was always about the best solution for my needs.

I just wish that I had bought it shortly after I found it and saved all the headaches the other methods gave me.