Frustration, pain are more appropriate terms from my pov than inflammatory. But ymmv.

It's like "Come on folks! You can do better than this! Please! Just stop! Think! Please. I'm begging you."

I'm not entirely convinced we should settle on ZFS just yet. It's fantastic and quite possibly the best option right now but it has a few limitations:

- The Linux implentation seems to have issues with releasing memory from the ARC back to the system

- The only way to expand a zpool is to add a new vdev. Pools are essentially a RAID0 of vdevs so if a single vdev fails, your entire server fails. You can mirror or RAID within a vdev but this means that the reliability of a vdev is the reliability of your entire pool. The problem here is that you can't just add 1 or 2 new disks since adding a 1 or 2 disk vdev would be data-suicide. For smaller servers, this is silly.

BTRFS looked like it was getting there but it proved buggy and unreliable. Personally, I'm waiting for bcachefs: https://www.patreon.com/bcachefs

If instead I ran RAID6, I'd have 80% of the disk available and I could add disks in single disk increments.

I think ZFS makes great sense for businesses that can throw money at disks but for smaller businesses or home servers it's kinda bad.

It was kind of a pain to configure (albeit quite flexible) but it's been pretty nice overall, already survived 1 disk failure and a capacity upgrade (during which I had to resilver after every individual disk upgrade, which was time-consuming, but after the last disk got upgraded, the extra space finally showed up)

Disk is cheap. There's no reason to design like it's not.

Plus, disk may be cheap but servers to house it are not (the kind of servers you'd run in your house, I know you can get cheap SC846 off eBay).

I'm sure it's not an issue in enterprise environments, but I've personally been bit by ZFS's inability to shrink or rebalance pools (even offline) several times now through personal use.

Those sort of use cases need to be handled if ZFS is to become a more general use filesystem.

tires++

;-)

Alright, the suspense is killing me: what OS is the one that gets it right? I ask this to "the rest of us" as you put it.

I'm not sure what reality you live in where Linux users like, actively deny things like lacking an appropriate COW filesystem with low latency, or things like the fact it took us a decade to get only close to DTrace. Your own mind? 6 month Ubuntu users? Internet forums where you talk to other BSD users and nobody else?

> Let the down votes begin.

Don't worry, I'm sure people will oblige.

However XFS is perfectly good for storage. I've used it on 15pbs worth of fileservers with no particular nonsense.

I wouldn't touch BTRFS with a barge pole, not because of any inherent stability issues (although that is a big factor) its the utter nastyness of the tooling.

ZFS is a joy to setup, and its simple, logical and the man pages tell you useful things.

BTRFS, not so much

1. filesystem is corrupted, once again

2. try to repair it

3. oh right, the repair tool can not replay the journal

4. try to mount it

5. admit after 3 hours of nothing that the journal-replay code triggered on mount actually can really not deal with corruption

6. reboot the server to get the filesystem unstuck

7. rerun repair, this time throwing away the journal

8. look at the empty filesystem with everything in lost+found

9. restore from backup

The team I'm in only runs 1'400 servers, yet this happens regularly.

But, mostly it happens because the fileservers dont have UPS's (The pipeline tools are almost exclusively COW, and backups are tested many times a week.)

I think we paid for support on XFS from either redhat or SGI, but I can't remember, I left that place a year or so ago.

I've never had the balls to run ZFS on large(100tb+) arrays. Last time I tried the way slab handling was translated from solaris cause many problems (but that was more than 4 years ago. ) Plus the support is a bit odd, you either go with oracle(fuck that), or one of the openZFS lot.

To get the best performance/stability you ideally need to let ZFS do everything, instead of letting the enclosure do the raid 6 (4 * 14 disk raid 6 with 4 spares). This of course is a break from the norm and to be treated with utmost suspicion.

Have you seen the GPFS raid replacement? thats quite sexy.

I run ZFS at home, because it works, and has bitrot checking.

Just as long as it does't get anywhere near capacity.

What a useless comment. If you are running out of capacity adding disks to a zpool is incredibly mind numbingly easy.

"zfs add poolname mirror disk1 disk2"

Every filesystem has performance hit the floor when you consume all available capacity. This is not unique to ZFS but to well, reality.

Both get you ZFS and something lighter-weight than VMs. Need Linux? Bite the bullet and use VMs.

So if you have the skill FreeBSD. Otherwise SmartOS makes a killer setup for containers unmatched by anyone else.

Maybe not so important in the container context, but sooner or later, somewhere you need persistence, and tape offers certain persistence features you pretty much can't get elsewhere, especially at its media price points.

(Plus, if they got this wrong, when it's really not that hard to get right (I've done SCSI at this level before), I wonder what else they have.)

No killing yourself necessary, really ;)