FlexRaid

Anybody ever tried FlexRaid?

I would like to build a homemade NAS, but I'm still in doubt how I would do that.

On one side, there's ZFS. To use this, I would use fine free programs like FreeNAS, NAS4Free (basically the continued development of FreeNAS 7 after it was sold by the original author) and ZFSGuru. The only thing holding me back here is that ZFS uses a relative big amount of disk space for parity and that it needs a LOT of memory if you want to use the more advanced features like deduplication (about 4GB/RAM per 1TB of storage)

On the other side there's unRAID. UnRAID has been there for a few years now, but it seems development is rather slow. On the other hand, it seems to be pretty stable and has a very active user base. One very big plus I think is that it was withstand the loss of one disk without loosing data, but should more than one disk fail, you still only loose the data on those failed disks: the data on the other disks can still be read by installing the disks into a different pc.

So I thought I would go for an unRAID system. It seems the best option there was for home users. The downside is: it costs if you want to use it with more than 3 disks (or want to use a cache disk to speed things up): $69 for use with up to 6 disks and $119 for use with a maximum of 21 disks.

But today I stumbled onto the page of SnapRAID. Though SnapRAID didn't seem for me because it's uses CLI, and is still pretty young, it did draw my attention to another product: FlexRaid. Now, this does look interesting! It's a system that can be used on both Windows and Linux. It offers both drive pooling and Parity Data Protection. It allows you to add drives later to the pool to expand it. These drives don't even need to be empty or formatted! Should it fail, you can recover the loss of up to 3 drives! AND you can still read the data on the disks if you pull the from the array and plug them into another computer. It's also not free (anymore), but it costs less than unRAID.

About the only thing I don't really like about it, is that it is only a Raid system and not a real NAS. The way I understand it , you can just as well install this onto your desktop PC. But I really would like a NAS to put things centralised on the network. On the other hand, I think this could be solved because it can be installed with OpenMediaVault as well. (this is the next project from the original FreeNAS author).

This FlexRAID thing seems incredibly eh... flexible More so that anything I've ever seen. Has anybody here ever used it?

Does anyone here actually understand what "Snapraid" does?
I don't. Their site seems verbose and uninformative.

Flexraid is similarly uninformative, but not quite as bad. It sounds just like unRAID to me, but with more marketing fluff and less detail. Can anyone here explain the difference between those two?

Thanks!

FlexRaid actually has a wiki that is pretty informative. I don't know exactly what how it differs from SnapRaid. Most likely SnapRaid is just a less mature product.

Well, if you understood anything from the wiki there, then please explain here how it all works.

How it all works exactly, I'm not sure. All I know is that it doesn't work on the block level like most RAID's do, but rather on the file level. In other words, a layer is laid over the existing file system and FlexRaid takes over. Using this method it's able to pool the harddrives into one big volume (like unRaid does, but easier since you don't have to preclear the drives and you can also add drive that already contain data), and it also offers redundancy by use of a parity drive (also like unRaid does). The main difference here is that you can choose how much redundancy you want. UnRAID is only able to use one parity drive, hence only one drive may fail. With FlexRaid, the amount of parity drives is not limited. If you wanted, you could use 5 parity drives in your array, meaning 5 drives may fail before you're in trouble. (and even then, only the failed drives will have lost their data, the other drives in the array can still be read by simply putting them in a different pc)

Obviously, the stress here is on pooling and redundancy, but this doesn't offer any speed increased like normal RAID (except RAID1) or ZFS does. On the other hand, it sounds very safe, easy and convenient.

Flexraid is not free if that's major factor (seems to be for unRAID).

I did look at this (including running demos etc on VMs) before stumping up the cash for unRAID. I can't remember what turned me off it now. It might come to me later.

Please do, I'd really like to know.
FlexRaid costs about the same as unRaid does, give or take a few dollars. In fact, if you want to use unRaid with more than 6 drives, it costs twice of what FlexRaid costs (and FlexRaid is not limited when it comes to number of drives). The cost is not a factor for me, because I was (and am) willing to pay for unRaid as well.

That's the problem. They don't explain how it works. So it is extremely difficult to judge whether it works, or works well enough to trust one's data to. Or is it any better than just ordinary RAID5/6 ?

Blind, vague claims about "file level" not "block level" are unhelpful. Digging deep into their wiki the disclaimers do start to appear, though, which is worrysome.

I personally wouldn't trust that software, at least not without information about exactly how they propose to protect my data.

Cheers

There are some worrying looking cases of "don't do this" in there aren't there.

I may be missing the point here entirely, but with current low cost of HDDs, what is wrong or not good enough with classic RAID5+spare(s)?

I've had my home server forever now, so I never tried to minimize the costs down to the sole storage/NAS, but I would think today a very inexpensive Windows or Linux box with RAID5+spares implemented via the OS would do the trick at low cost?

No?

RAID arrays don't deal well with power failures, and take forever to rebuild/self-check when used with modern multi-terabyte drives. And when Mr.Murphy brings down the Law, you lose *everything* from all drives, not just the failed ones.

So this discussion has been exploring alternatives to the dusty old RAID model.

But were I to do RAID (never again!), I'd build it from a generic Linux box.

The danger with RAID with larger drive sizes is that rebuild times take longer. The pace of capacity growth has far exceeded the pace of drive speed increases.

Whenever a RAID5 array experiences a failure, the entire rest of the drives must read every single sector correctly to reconstruct the failed drive. And since RAID is a block level operation with no concept of the filesystem, no shortcuts can be taken in unused space. If hard drives had TRIM support, this could help reduce RAID rebuild times.

The larger the array, the higher possibility of a second drive failure during a rebuild. Especially if the drives are from the same manufacturing batch, and suffer a similar fault due to a worn out component.

The really bad situation is when the RAID software or hardware never does a health check of the array. It's possible a bad block has been sitting on some drive, undetected because the file hasn't been read in months. Another bad block on a different drive comes up and is seen due to it being active, the RAID fails the drive, then explodes when it hits the earlier undetected bad block. For ReadyNAS users, make sure "Disk Scrubbing" under Volume options is on to avoid this. SmartArray cards from HP default to doing passive checks when active array use dies down.

Some of this risk can be mitigated. RAID 1+0 results in far less drive activity across the entire array to rebuild a failed drive. RAID 6 offers the ability to recover even if 2 drives tank before a complete rebuild. But it can still be a gamble with larger SAN sized installs.

These alternative redundancy models sometime offer benefits, such as being file based instead of block based. This will ensure repair of a bad disk runs much quicker.

TRIM (aka. "Write-Same") support can help with initial build of a RAID (zeroing all of the disks), but I don't see that it aids a resync at all.

Cheers

Could it not help speed things up, as the drive will know which sectors are blank without actually reading them ?

No, other than SSDs, drives don't "remember" that info. And for a RAID, it's really the host/RAID software that needs to remember it, to avoid the reads.

So... if RAID is not a backup, and your data (all your data!) in a RAID array is at risk due to the non-trivial possibility of a second failure during extended multi-terabyte restructuring, and it takes less time to recover your data from a backup than to restructure an entire RAID system...

Can someone please tell me what conceivable reason there is to have a RAID array? I have a drawer full of backup disks, plus a second set of backup disks stored off-premises down the street at my neighbor's house.

Yes, I know, with a RAID you can continue working uninterrupted if a hard drive fails. But is there anyone on this bbs that does that? Or do you immediately shut down and replace the missing disk and then wait nervously for half a day while the system rebuilds itself? Whose work is so time-critical that they can't take an hour off to copy over the data from a single backup disk?

Not mine, that's for sure. But... YMMV.

tanstaafl.

I know they don't, Tom was theorising on the benefits to RAID if hard drives id have TRIM support. And if they did (along with a FLASH chip to remember the info) then you could see how it would speed up rebuilds on an array with lots osfempty space (as the drives could rapidly return zeros for blank sectors without having to read the actual hard drive).

I have a massive movie collection, mostly from storing the full raw DVDs on my NAS for later playback. This takes up about 5TB worth of space, and no single hard drive holds that much data. RAID allows me to combine hard drives into one massive storage pool.

I also don't see the need to back this up, since the DVDs are still around. RAID gives me a little safety net in that I won't have to rerip all the DVDs if one hard drive tanks.

RAID is also a performance boost in most cases, allowing heavy I/O workloads to be met.

The reason we constantly say RAID is not a backup is that people sometimes believe it is, and save their only copy of some data to a RAID.

Oh, and most RAID systems can rebuild while still remaining online. So even if failure strikes, someone can keep working on the system while it recovers with just a little bit of a performance hit. My last job had everyones desktop set up with RAID for both performance and the ability to keep them working if one drive failed. Recovery from a complete drive failure would take more then just a hour in that environment. OS load, + bunch of apps + restoring work from the version control system would easily chew up a productive day.

For home users like you and I, none.
The main use seems to be as the default method for pooling many drives into a single large filesystem, for simplicity, so that one doesn't have to bother balancing storage use among individual drives.

But unRAID and mhddfs also solve that problem, without the same risk of losing (or having to restore) everything that comes with RAID5.

Here, I swore off RAID years ago, when large hard drives were still just a few hundred GB in size. With them now in the 2-3TB range per drive, there's just no frickin way I'd want to put up with a RAID trying to manage those sizes. Too slow to build, too slow and risky to resync, low tolerance of failure, and a real risk of losing everything. And very few people I know actually back up their RAID array.. not even Julf (who knows better!).

So instead of adding drives to the RAID for redundancy, further complicating things, I use a pair of mhddfs arrays. One is live, the other is a complete duplicate (aka. "backup"), that gets resynced periodically.

And anything I deem "critical" has several additional backups.
That's my system, and it works for me. It won't work for you, because MS-Windows doesn't have "mhddfs".

Cheers

Depends on the home

Our boiler room hosts 3 servers - one provides "general services" (internal web pages showing stuff like weather data, power consumption, security cameras etc.), one runs house automation and phone system, and one is the media center. All 3 run RAID just to ensure availability/uptime.

So those ought to all be RAID1 systems, right?
Or at least the first two, if not the third one.

Yup, raid1, raid1, raid5 (for now)

Peachy. I actually *like* RAID1 (who doesn't?).
RAID10 ain't half bad, either.

Oh, ok, sure. I did miss the point of the thread.
I also agree that RAID-5 id definitely not a good solution for home, and I see why looking into something better.

I've been using RAID-1 (in addition to backups) forever and would not consider either today RAID-5.

Windows 8 "Storage Spaces" partly addresses that, at least in theory. It allows redundancy and volumes across physical HDD at the file level, and supposedly HDD removed from a Storage Space pool are readable independently. I have not tested yet this myself though. Documentation so far is still inconsistent as this is iteration 1.0 of "Storage Spaces". I'll test soon, in any case, out of curiosity.

I thought that the new storage tech in Windows 8 was an evolution of what Windows Home Server had. Or did they start from scratch? I'm not as well versed in Windows storage tech as I once was.

Supposedly, they took only some of the features of WHS "Extended Drives" or whatever they called it back then (I never used that myself), and re-engineered from scratch making it available to both W8 and Windows Server 2012. So, under the hood, this should be a more solid and reliable server class feature than before.
Again, I never tested it directly yet, so I am not sure if, as I would expect, there are also different sets of features between the server world and the desktop world.

Indeed, if well designed, Storage Spaces could actually be useful in homes too, for the very reasons at the origin of the questions asked in this thread.

But with its current implementation (on paper) what leaves me very puzzled is that Storage Spaces lets you do things that are quite crazy to me, such as adding USB 2.0 drives to a pool of (e)SATA or SAS drives, for example, which can even make sense in some very peculiar situations if done by somebody aware of what is happening, but in most cases that simply means killing performances. I see that the idea in mind is plugging an external, USB based storage such as Drobo permanently (and still don't like it much), but I can also picture somebody doing such things and STILL treating USB drives as removable... Or, letting you add significantly smaller drives to a pool of larger disks, which would make no sense in most cases (all cases?).
I mean, on paper it is even TOO flexible if targeted also to homes, as it is. Nothing that would be dangerous for people in here - and all reversible stuff, I read - for example, but I can picture in my head so many friends who would just run into trouble or confusion for the simple reason that they would be acting according to the principle "add more drives to add more space" and plug an old 200GB drive to their 3x 1.5TB drives pool.

in any case, I have not even seen the GUI yet, other than in documentation.

Why would you think people would do this? I'd think they'd be more likely to simply think "I'm out of disk space; I'll just buy an external 2TB USB disk and throw it on there".

I certainly used to do this when my Linux box (a nettop with only a single internal bay) was doing my NAS duties.

I think that it's only people like us that tend to have 200GB disks lying around the house.

Maybe, I don't know.
I do know people with old disks in the house that keep thinking they can re-use them someday. Often they are old USB disks.
I suspect those are the disks they'd be adding to a pool "in emergencies", that is when all at the sudden they realize they need few extra MB for whatever they're downloading in that very moment.

Interestingly, you may add SD or CF memory cards to a pool! I like that you can, but I am a bit uncomfortable thinking of the possible misusage...

This is not true, AND it's the big difference between the Drive Extender of WHS and Storage Spaces. Sure, the result is the same, a big pool of disk space, but they did start from scratch when they gave up Drive Extender. With Drive Extender, it was possible to remove a disk from WHS, but it in an external enclosure or a different Windows system and the disk would be readable. This is no longer the case for Storage Space, which makes it a BIG risk IMO. AFAIK, there is also no redundancy with Storage Spaces, only drive pooling, making this even more dangerous.

Also, since it's a new technique, data recovery software hasn't been developed for for, so if Murphy hit, it'll hit hard! I've actually read a story where one user of SS had a failure and contacted Microsoft about it, and he actually got the message to put the drive in storage somewhere for a couple of months, until data recovery software had caught up with Storage Space and it would be able to retrieve data from it. :$

If that is the case, that's a big step back from DE. However, I did not read anything clear on this in their very documentation (hence my "supposedly"). Have you actually tested this? My guess is that a drive is not immediately readable but it becomes so after few clicks in Disk Manager. Just a guess.


There actually is. It can be configured up to double drive redundancy. This is clearly stated by MS. I think SS would not even make sense without redundancy.

RAID5 is indeed too dangerous to use, for the reason Tom has explained. That's why a lot of 'hardcore' home users started using RAID6, where two drives may fail. This is course doesn't help with the slow rebuild times and scrubbing etc... It also means you'll need at least 4 drives and a lot of drive space simply for parity calculations so you won't have that much free space left. (if you only use 4 disks, you'll loose half of your disk capacity, which makes it rather costly)

If I was to use a 'regular' RAID system, I'd use ZFS (which isn't 'regular RAID' in any sense, but you probably get my drift). ZFS allows for up to triple parity, but here also double is more than sufficient. The big plus about ZFS is that it's self-healing, is a copy-on-write filesystem and because of this can use snapshots which are very handy.

The only reason I didn't start using ZFS is because, as I understand it, it's not possible to add disk space to a created RAIDZ array in a drive pool. It is possible though to add disks to the drive pool and to create additional RAIDZ arrays, but that's not exactly what I want. Like most users, I simply want to see one big volume, containing all my disk space. The big plus of ZFS is it's robustness and it's speed.

But of course there is a downside to ZFS as well, it being the demands of ZFS are pretty high if you want to use it to the best of it's capabilities, especially the memory demands (and this is especially true if you want to use de-duplication). On the other hand, it allows you to use an SSD cache drive which speeds things up considerably. But on the other side, I use a gigabit network at home. The theoretical maximum speeds is 125 MB/s (if I don't use any link aggregation, which I could do but don't). So if the server it capable of providing that speed, it's sufficient, because I don't think I'll benefit much from having SSD-type speeds available if the network can't keep up with it.

So all this has led me back to unRAID. But now I start reading stories again of 50 hours of 'preclearing' drives (making new drives ready for use with unRAID - a mandatory part of the procedure) before you can start to use it, or days to calculate the first parity calculation (so it takes days before you are somewhat protected). Also, the speed of unRAID isn't all that good. I read about speeds of 40-50 MB/s which I find not very impressive.

In other words - I'm still not quite sure which route I'll take. My demands are rather simple though: I want a NAS system which will provide me with speedy file transfers (+100 MB/s), drive pooling (with easy expansion when full) and some form of protection (don't worry, I'll still keep a separate backup as well). As far as I can tell, there is no do-it-yourself software available yet which gives me all those options. Commercial NAS'es like Synology also are not an option, both because of their price and because they still use old RAID technology and no drive pooling.

No, not tested it myself because I don't want to be the guinea pig. But you can easily find horror stories about these new SS, just Google for them. It seems SS is still very young technology which may get better with new revisions, but for now, it's probably not a good idea to use.


You're right, I take that back. There is redundancy, like there was with Drive Extender. You can choose to mirror the data or use parity. But unlike with Drive Extender, it's no longer possible to mirror only one specific folder. This was very handy: stuff that was important you mirrored, less important stuff you didn't. The benefit of this was clear: you saved disk space. Since you cannot do this with SS any more, you'll need a lot more disk space for redundancy. Of course, this all is of little help when the entire SS goes bad. This is known to happen now and then, so much even that I wouldn't risk using it for now, not until there is an option to actually recover data from a failed SS.

How big a volume would you like to get, to begin with?


Edit:
Horror Stories: I've been looking around the web, but all I read was pretty trivial mistakes, I think. I am not sure it is a too young technology at its core features (reliability above all). It rather seems to be so in terms of GUI and user experience. But again, these are just words, I have no direct experience with SS. Two or three old USB disks should be enough to answer few basic questions, though. I'll post what I find out.

The thing about "4 drives" is that this is often the limit for an enclosure or chassis. Sure, there are larger (and smaller) units, but four is pretty common.

So with four drives, just use two of them straight-up (or with mhddfs), and then use the other two as 100% backup copies.

So two data drives, and a full backup. Obviously better than RAID6 w/o a backup, and better than RAID1, because errors/deletions don't get mirrored until you tell them to get mirrored.

Which is exactly what I'm doing, except it's five drives: three and two.

C:> and D:> backed up to an internal E:> drive; and F:> backed up to an external L:>.

G:>, H:>, and I:> are used for temporary USB plugins (Ipod, flash drive, camera, whatever) and J:> and K:> are external backups saved off-premises.

It works for me. YMMV.

tanstaafl.

I have had a server always on at home since the days of Fidonet BBSs (I used to run one), and so I am used to have a machine always on in my home that I can rely on.
Currently, my home server hosts few websites, my data, my gf's data, my picture and music collection, and more stuff.
Since I don't keep (yet) a video collection in it, 2TB seem to be enough so far, even though my recent interest in photography is (not so) slowly pushing me to considering an expansion.

In any case, I have a RAID-1 storage system including system and data volumes.

In addition to that, I have three more "on-line" 2TB disks that I use for automatic daily, weekly, monthly backups, which, over time, I found are more than enough to address any deleted-by-mistake issue.

And finally, I have two "off line" USB disks I use for manual backup.

This has worked very well so far. I keep considering an on-line backup but I never got myself to getting the whole system setup, even though I like Crashplan. My bad.

Now, besides the obvious advantage of eliminating the down time in case of a system disk failure and consequent restore from backup (I never gave up the mentality of having the home server up and running at all times, even though in these days that is not as important to me as it used to be), the RAID-1 allows me to expand my drives quickly and with just 2 reboots (as I don't have hot plug storage in my case). If it wasn't for that, I would probably reconsider a simpler model.
On the other hand, my home server also serves a purpose of testing, experimenting, and fun, so some "unnecessary" redundancy is welcome anyway.

In fact, as Mark here recommends a good 3TB disk, I'll probably get two. Also because people in here convinced me that RAW pictures are obviously a good thing to keep.

Unless the picture sucks. No point in keeping the RAW file for a blurry picture.

Oh, surely you're mistaken.

Just last week on one of the CSI television shows, I saw them digitally enhance a low-resolution picture from an ATM security camera so that they could read the license plate on the car of a fleeing felon that was six blocks away. In the dark. During a rainstorm. At 80 miles an hour.

Surely you could accomplish the same thing with Lightroom or Photoshop.

Couldn't you?



tanstaafl.

Actually... Time, and good algorithms,may prove you wrong sooner than later!
I read something about it on dpreview a while back, and Google seems to return some software titles, already available. Admittedly, I am not so sure a RAW file is needed for that.

https://www.youtube.com/watch?v=KUFkb0d1kbU

True enough -- de-blur is a pretty good PhD topic, and I have seen some pretty decent results from some of those systems. If your blurry shot is the only shot you've got, it might be worth keeping.