News for hackers. Stuff that matters.
Re:OpenZFS Merged to FreeBSD by ggm on Tuesday August 25, 2020 @01:18AM (#24268309)
It is amazing that two disparate OS communities can take a shared codebase and integrate it deeply into their structure.

There are many places where it would be almost impossible to achieve this kind of share. Witness the 'compatability library' work which exists to avoid ABI blackholes running one platforms code on another platform.

BTW I have been able to carry FreeBSD ZFS LUN on a NetAPP and mount them on Debian ZFS, and vice-versa, since I started using ZFS. What we got here is better code alignment but with qualifications, ZFS has always been a common platform (with fringing differences)

I never had illumos or Solaris, I cannot comment on how well it would have worked.

Re:OpenZFS Merged to FreeBSD by kev009 on Tuesday August 25, 2020 @02:14AM (#24268535)
Brian Behlendorf and Matt Macy are both fantastic engineers, other people contributed but this is ultimately the kind of thing that must be led by a small number of (in this case two) extremely senior engineers working together for a common outcome with raw technical ability to get everything off the ground and prove it works; and refined leadership skills to stick around til it's actually done!
Re:OpenZFS Merged to FreeBSD by nosequel on Tuesday August 25, 2020 @02:21PM (#24274333)
Matt (Kip) Macy also showed "refined" leadership when he

> told workers in September 2006 to cut the beams that supported (his tenant's) apartment's floor... They also shut off Morrow's electricity, cut his phone line and had workers saw a hole in his living room floor from below, prosecutors said.

So yes, he has really refined leadership qualities!

https://www.sfgate.com/bayarea/article/S-F-landlords-charged...

Re:OpenZFS Merged to FreeBSD by kev009 on Tuesday August 25, 2020 @09:06PM (#24278014)
illumos concern trolling, this has nothing to do with engineering leadership and is just lashing out a bad faith personal attack because you are friends with joyent people. Being unable to acknowledge that he served his sentence and is reformed indicates you are just hung up on a failed project that is as useful as spilled milk.
Re:OpenZFS Merged to FreeBSD by athms on Tuesday August 25, 2020 @09:52PM (#24278252)
Somebody that continues to blame the victims for their troubles is NOT reformed.
Re:OpenZFS Merged to FreeBSD by kev009 on Tuesday August 25, 2020 @09:57PM (#24278288)
Like Bryan Cantrill?
Re:OpenZFS Merged to FreeBSD by tal8d on Tuesday August 25, 2020 @03:40PM (#24275185)
Looks like the CoC mob has failed to run off a capable developer... for all we know that tenant might have one day contributed code - this is a gross violation of "applicable spaces" clause!
Re:OpenZFS Merged to FreeBSD by gjm11 on Tuesday August 25, 2020 @04:01PM (#24275459)
Whatever does this have to do with codes of conduct? I mean, maybe software projects should have CoCs that say "do not do things likely to kill your tenants", but normally they're concerned with conduct within a project, and in any case "the CoC mob" seems to indicate you're somehow arguing against having and enforcing CoCs. Is there some background here that makes your comment make more sense than I'm currently able to get out of it?
Re:OpenZFS Merged to FreeBSD by tal8d on Tuesday August 25, 2020 @05:41PM (#24276498)
> Whatever does this have to do with codes of conduct?

Whatever does a rental property issue have to do with OpenZFS? What about political contributions? Are you genuinely unaware of all the crazy witchhunts, totally unrelated to the code base, that have occurred as a result of this CoC push? Have you ever heard of ESR? http://esr.ibiblio.org/?p=8609

Re:OpenZFS Merged to FreeBSD by gjm11 on Wednesday August 26, 2020 @06:16AM (#24281032)
> Whatever does a rental property have to do with OpenZFS?

Not much (though I think it's fair to say that when someone is praised for their "refined leadership" it's talking about more than just the quality of their code). In any case, "that other guy also said something irrelevant" isn't really to the point.

> What about political contributions?

What about them?

> Are you genuinely unaware of all the crazy witchhunts [...] this CoC push?

My confusion is because so far as I can see "this CoC push" has nothing to do with anything here, it's just something you want to talk about. The OP isn't about CoCs. The thread you replied to isn't about CoCs. The comment you replied to isn't about CoCs. Kip Macy's mistreatment of his tenants isn't about CoCs, or caused by CoCs, or prohibited by CoCs. The OpenZFS merge isn't about CoCs. (Well, quite possibly the OpenZFS project and/or the FreeBSD project have them, but so what?)

> Have you ever heard of ESR?

Yes. I don't look to him for political guidance, though.

(Two other things, besides codes of conduct, that have nothing to do with anything in this discussion aside from your apparent wish to talk about them: political contributions, and Eric Raymond.)

Re:OpenZFS Merged to FreeBSD by nosequel on Tuesday August 25, 2020 @06:08PM (#24276789)
To be fair, I was originally replying to the complements about how amazing Kip Macy is, not OpenZFS. If the parent comment was about OpenZFS, I wouldn't have replied. I thought posting about Macy's multiple felony counts is a solid response about how amazing he is at leading folks. Also gtfo with your anti-CoC outrage with this. He didn't get caught using the wrong pronoun in a code comment.

> Kip Macy is accused of three felony conspiracy charges, three burglary charges, two stalking charges, two grand theft charges and one felony count of shutting off service

Re:OpenZFS Merged to FreeBSD by athms on Tuesday August 25, 2020 @09:57PM (#24278291)
That is why Kip goes by his middle name now. He also moved from California to Washington.

This was big news in SF Bay Area when it happened. It was FreeBSD's Hans Reiser moment.

Another gem, Kip kicked one tenant in the chest while he was sitting on the couch and threatened him with a gun.

I am curious if he ever reimbursed his parents after they drained their retirement savings and offered their house as collateral to pay for his $500k bail, which was forfeited after Kip and his wife Nicole decided to skedaddle to Europe instead of appearing in court. Kip continued to work on FreeBSD while on the lam in Italy with full knowledge of the core team, with many members actually defending his behavior. Kip and Nicole also never showed for the civil trial and lost by default.

The original plea deal offered by the DA, before they absconded, was 6 months in county jail with charges reduced to misdemeanors. Instead, both were sentenced for 4 felonies and got 4.5 years (reduced for good behavior), with Kip ending up in San Quentin -- a maximum security prison housing California's death row.

I remember ABC News did an interview with Kip while he was incarcerated and he blamed his tenants for all his troubles. At least he can no longer (legally) possess a gun since he is a convicted felon.

Re:OpenZFS Merged to FreeBSD by senderista on Wednesday August 26, 2020 @01:41AM (#24279492)
This is indeed Hans Reiser-level drama. After reading about their shenanigans I can't believe anyone would choose to work with such a person.

https://abcnews.go.com/US/exclusive-landlord-hell-defends-te...

Re:OpenZFS Merged to FreeBSD by athms on Tuesday August 25, 2020 @09:56PM (#24278281)
So that is what Kip was doing in Italy while being on the lam from the law. He is really dedicated. I wonder if San Quentin allowed him to code while he was incarcerated.
Re:OpenZFS Merged to FreeBSD by znpy on Tuesday August 25, 2020 @03:17AM (#24268848)
Well you're citing netapp, zfs, freeebsd and debian. Those are all top notch.

In all honesty I can't understand why would anyone use anything other than zfs nowadays for important data.

Re:OpenZFS Merged to FreeBSD by retro64 on Tuesday August 25, 2020 @11:32AM (#24272397)
In all honesty I can't understand why would anyone use anything other than zfs nowadays for important data.

At a personal level, I opted out of it based on simplicity and familiarity. If I have a data drive where I need to move and mount in another box, I do not want to mess with the complexities of supporting ZFS to get at that data or the risk that the only OS I have available can't read it.

Again, this is a non-commercial environment, but I consider my data important as well. I've decided to go the route of keeping multiple backups of my data spread across multiple drives. I've already experienced bitrot several times over the years, but for me, this approach is more practical than relying on ZFS.

Re:OpenZFS Merged to FreeBSD by radiowave on Tuesday August 25, 2020 @11:51AM (#24272582)
I don't mean to be criticizing your choices, but I'll note here that if you're willing to avoid enabling the latest and greatest ZFS features, bringing up a machine that can mount your disks can be as simple as booting from Ubuntu install media, and having internet available so you can run "apt install zfsutils-linux".

(And somebody's probably going to correct my ignorance and point out that beyond some release 'x', the ZFS stuff is already built-in.)

Re:OpenZFS Merged to FreeBSD by retro64 on Tuesday August 25, 2020 @12:09PM (#24272803)
Is it that easy now? I actually was in this situation about 3 years ago and found I could not simply mount the ZFS drive - and I cannot recall exactly why. I think it had something to do with the pool…configuration(?), and I went down a rabbit hole trying to sort it out. Fortunately in this case the tradeoff in time was more valuable than the data (it was a system drive on a small SSD) so I had the option to ditch my efforts with little to lose. But, I did spend a handful of hours on it with no traction.
Re:OpenZFS Merged to FreeBSD by bpfrh on Tuesday August 25, 2020 @12:46PM (#24273269)
I was able to transfer a zfs pool from a linux server to a freebsd server with different hardware simply by moving the physical disks to the new server.

For me ZFS has been rock solid, even with power cuts during running vms(linux and windows) and a scrub.

I even managed to steal away enough ram from a server so that ZFS produced a stacktrace in dmesg, no data corruption, even on running vms.

Re:OpenZFS Merged to FreeBSD by znpy on Tuesday August 25, 2020 @12:57PM (#24273392)
Yes it is :)
Re:OpenZFS Merged to FreeBSD by asveikau on Tuesday August 25, 2020 @12:18PM (#24272936)
I've used FreeBSD install media to recover too. You can get a shell and there's no need to download other stuff.

I haven't done this to recover ZFS as written by Linux or Solaris/illumos, not sure how well that works but wouldn't be surprised if it does.

Re:OpenZFS Merged to FreeBSD by traceroute66 on Tuesday August 25, 2020 @06:22AM (#24269822)
@znpy

Why people aren't just using ZFS (or BTRFS) ?

Well...

Maybe because XFS is well proven in certain critical environments, especially ones which might involve frequent manipulations of large quantities of small files (think mailserver or database). Indeed, some developers will tell you in the docs that they only support XFS or ext4 and that you're on your own if you choose something else.

Maybe because ZFS and BTRFS still have their bugs, small and big (hello RAID5 in BTRFS ;-) ) and can be heavily dependent on particular implementations.

Maybe because in a virtualised environment, you loose many of the benefits of ZFS or BTRFS (they like to see the raw disk, not some abstract notation).

Maybe because the combination of XFS/ext4 plus LVM is more than good enough for most people as that route still provides snapshots etc.

Maybe because BTRFS needs babysitting[1] (don't know about ZFS, I suspect it might).

Maybe because not everyone feels the need to "keep up with the Jonses".

[1] https://www.youtube.com/watch?v=8YUC-r1aXAc

Re:OpenZFS Merged to FreeBSD by linsomniac on Tuesday August 25, 2020 @11:13AM (#24272165)
As far as any of your reliability concerns, my experience with ZFS has been that it is amazingly reliable.

If I had to pick one place to put precious data that I needed to access for 10-20 years, ZFS would be it. Of course, I wouldn't put it in one place, so if I picked places to put that data, ZFS would definitely be one of them.

I've been using ZFS for over a decade, both for my personal storage systems and for work (largely backup servers), and despite heavy use and abuse, I've never once had data loss while using it. I've had some white-knuckle times, but in the end ZFS was able to recover all the data.

I say "abuse" because I was able to write a stress test program based on my usage that uncovered several ZFS+FUSE issues early in that port.

Just a data point.

Re:OpenZFS Merged to FreeBSD by radiowave on Tuesday August 25, 2020 @11:59AM (#24272672)
Likewise. Been using ZFS for over 10 years. Been through intermittent controller faults, backplane faults, cabling faults on a variety of ZFS, ext, and NTFS systems.

The one time I lost data from a ZFS system (which was a faulty controller silently writing garbage to multiple disks in the array), ZFS told me exactly which few files were uncorrectable, and I was able to restore them from backup.

Not one single second of doubt as to which storage system I want my data on.

Re:OpenZFS Merged to FreeBSD by masklinn on Tuesday August 25, 2020 @11:39AM (#24272460)
That you're equating ZFS and BTRFS, and tarring ZFS with the issues of BTRFS, clearly shows how much you don't know about it. ZFS was production-stable more than 10 years ago.
Re:OpenZFS Merged to FreeBSD by znpy on Tuesday August 25, 2020 @01:00PM (#24273432)
This is the main point most people just refuse to accept out of blind religion: zfs was production stable 10 years ago and is production stable TODAY.

With btrfs you're still waiting for stuff to be properly implemented.

Re:OpenZFS Merged to FreeBSD by danudey on Tuesday August 25, 2020 @04:25PM (#24275709)
> With btrfs you're still waiting for stuff to be properly implemented.

Redhat gave up on waiting, and is removing btrfs as a supported option in RHEL. Mostly, I assume, because the btrfs team has yet to ship something anyone in enterprise would be comfortable betting their customers' data on.

Re:OpenZFS Merged to FreeBSD by fomine3 on Tuesday August 25, 2020 @08:28PM (#24277829)
RedHat is not "just waiting" company for core component but hiring maintainers and strongly contributing company if needed. it's happened on XFS adoption.
Re:OpenZFS Merged to FreeBSD by freedomben on Wednesday August 26, 2020 @10:12AM (#24282627)
RedHat is actively working on Stratis too (which I'm excited for): https://stratis-storage.github.io/
Re:OpenZFS Merged to FreeBSD by xxpor on Tuesday August 25, 2020 @12:02PM (#24272708)
I think the biggest issues with using ZFS as a linux user come down to

A) ZFS-on-Linux was always behind BSD ZFS until recently.

B) The license issue, so you have to compile it as a separate module using DKMS, which means having a compiler, the linux headers, etc etc. No where near as low friction as just choosing ext4 or btrfs.

Re:OpenZFS Merged to FreeBSD by derefr on Tuesday August 25, 2020 @12:12PM (#24272844)
> ZFS-on-Linux was always behind BSD ZFS until recently.

Sure, but the sort of people who need ZFS are the same sort of people who are doing their storage on a SAN, and so the sort of people for whom choice of OS (for the SAN itself) can be contingent on choice of storage layer, rather than the other way around.

This is half the reason ZFS-on-Linux didn't have much momentum: most ZFS deployments were happy to just run BSD on their SAN, then serve SAN volumes out over the network (or into a hypervisor cluster) for Linux systems to consume.

Re:OpenZFS Merged to FreeBSD by andoriyu on Tuesday August 25, 2020 @02:06PM (#24274146)
> ZFS-on-Linux was always behind BSD ZFS until recently.

FreeBSD was never really working on ZFS in a way ZOL was. Whole reason for the switch is because main contributor upstream used by FreeBSD stopped using ZFS.

ZOL had more features for a while now as well.

Re:OpenZFS Merged to FreeBSD by asveikau on Tuesday August 25, 2020 @09:06PM (#24278017)
I think there is a reason the part you quoted was written in the past tense. The situation has changed. But these things have momentum and people might have adopted FreeBSD years ago because of the former status quo, where FreeBSD had it integrated in the default install and Linux distros did not.
Re:OpenZFS Merged to FreeBSD by tw04 on Tuesday August 25, 2020 @01:19PM (#24273656)
I think it's a disservice to even mention btrfs and zfs in the same breath. ZFS is a battle tested, enterprise ready, production grade filesystem and lvm.

BTRFS is a science experiment by comparison. They tried to copy zfs features because Oracle won't release it under GPL and did a fairly poor job of it. The fact we're this many years later and their RAID5 code still has total data loss bugs pretty much sums up btrfs.

Re:OpenZFS Merged to FreeBSD by danudey on Tuesday August 25, 2020 @04:30PM (#24275748)
Anecdotal, but I had a power outage at work last year and my relatively freshly-installed 18.04 system, for which I chose btrfs as the root filesystem, completely corrupted.

I managed to extract most of my data to another disk on another system, but it was pretty shocking. I haven't seen unfixable filesystem corruption like that since ext2 in 2001. I'd completely forgotten it was a thing, but there I was in 2019 trying to use a second linux box to salvage my data.

Of course, whenever I mention this the btrfs zealots always come out of the woodwork saying "you must not know what you're doing if you broke btrfs" or "you obviously didn't really try if you couldn't get it fixed" or a half-dozen other excuses, but in the end I've just completely given up on btrfs. It's just not worth the hassle.

Re:OpenZFS Merged to FreeBSD by necheffa on Tuesday August 25, 2020 @08:12AM (#24270518)
Do people really create zpools within a VM for more than playing around? The best practice is to point a VM at a zvol the way you might use LVM on bare metal to back storage for VMs.
Re:OpenZFS Merged to FreeBSD by ianai on Tuesday August 25, 2020 @10:24AM (#24271637)
Right. The storage pool for VMs should be zfs (if you’re going to use zfs as an assumption) while the VMs should be something else unless you’re doing something specific to use zfs in the VM. Zfs is engineered to almost turn a system into a file system server in exclusion so use it for its strengths appropriately.
Re:OpenZFS Merged to FreeBSD by znpy on Tuesday August 25, 2020 @12:58PM (#24273412)
Big thumbs up for xfs!

It's just... Snapshot thing and replication (zfs send) are so damn easy in zfs...

Re:OpenZFS Merged to FreeBSD by danudey on Tuesday August 25, 2020 @04:22PM (#24275668)
I would avoid lumping BTRFS in with ZFS. The two are very different.

> Maybe because XFS is well proven in certain critical environments, especially ones which might involve frequent manipulations of large quantities of small files (think mailserver or database). Indeed, some developers will tell you in the docs that they only support XFS or ext4 and that you're on your own if you choose something else.

XFS is great, except it doesn't do any of the things that ZFS does well. For example: snapshots (to the fantastic extent that ZFS does), multi-volume support, shared storage pooling, etc. The only thing XFS offers that ZFS doesn't is support for reflink copies on Linux (i.e. "cp -R --reflink=always src dst" for near-instantaneous copies of large directories with no extra disk usage).

> Maybe because ZFS and BTRFS still have their bugs, small and big (hello RAID5 in BTRFS ;-) ) and can be heavily dependent on particular implementations.

ZFS has been, in my experience, amazingly bug-free over the last however long I've been using it (5 years in production, more personally). ZFS has working and reliable RAID support; only BTRFS is lacking it.

BTRFS, on a stock Ubuntu install with default settings, corrupted after a power outage, and could only recover most of my data, but not all. I haven't had filesystem-related data loss since ext2 in 2001.

Also, for everyone except people running Solaris, there is (now) effectively one implementation, and it works fine.

> Maybe because in a virtualised environment, you loose many of the benefits of ZFS or BTRFS (they like to see the raw disk, not some abstract notation).

You still get point-in-time snapshots on a copy-on-write filesystem, which can support different block sizes per subvolume, on-the-fly compression, and incremental or full exports, all from a shared, extendable storage pool. Those alone are huge enough to justify it for me.

ZFS wants to see the underlying disks so that it can make sane judgements about block allocation, alignment, etc. On a VM, that shouldn't matter, because your host filesystem/SAN/etc. should be doing that anyway.

> Maybe because the combination of XFS/ext4 plus LVM is more than good enough for most people as that route still provides snapshots etc.

LVM snapshots are awful from a performance standpoint.

In ZFS, when you create a snapshot, any new data is written to new blocks (read the old block, make the change, write it to the new block), so your snapshot points to the old blocks on disk.

From what I can tell[1], LVM snapshots mean that when you write data to a block, LVM reads that block, writes it somewhere else, and then updates the original block in-place. This makes every write a synchronous write, because you have to write the old data to its new block, sync to make sure it actually gets stored, and then do your new write.

BTRFS snapshots cannot be recursive (i.e. you cannot snapshot /data/ and /data/gitlab and /data/svn and /data/backups atomically). They argue that this is a feature, but I would consider it a massive bug. LVM snapshots by their nature are not recursive because LVM has no concept that I can find of nesting.

> Maybe because BTRFS needs babysitting[1] (don't know about ZFS, I suspect it might).

ZFS, as far as I can tell, does not need babysitting. I used it for storing backups at my last job; we had a QNAP NAS exporting over iSCSI, ZFS was using the iSCSI storage, and we wrote data to it. Nothing ever went wrong, and I literally never checked on it unless the power went out or something of the sort. It was three years before I ever got around to setting up monitoring for it, because literally nothing ever went wrong so I completely forgot that there was anything to monitor. Eventually I did set it up, but it never went off. Literally set it and forget it.

In summary: ZFS is fantastic, and basically never needs to be taken care of. It does its thing and it works great, and that's it. BTRFS has corrupted itself arbitrarily after a power outage (which I thought we'd solved with ReiserFS back in 2001), LVM snapshots are slow and awful, and the tooling for LVM and BTRFS is a usability disaster while also losing a lot of features that ZFS has (like on-the-fly compression, block deduplication, etc.).

Re:OpenZFS Merged to FreeBSD by fomine3 on Tuesday August 25, 2020 @08:33PM (#24277853)
ZFS is great but CoW makes performance bad on some situation in theory. So there are some situations better to use XFS (without using LVM snapshot all time).
Re:OpenZFS Merged to FreeBSD by lousken on Tuesday August 25, 2020 @07:14AM (#24270125)
if i remember correctly LVM snapshots were deemed to be lowering performance of a system if you had lots of them - is that still a thing?
Re:OpenZFS Merged to FreeBSD by post-factum on Tuesday August 25, 2020 @07:25AM (#24270197)
Thin LVM snapshots solve that.
Re:OpenZFS Merged to FreeBSD by andoriyu on Tuesday August 25, 2020 @02:02PM (#24274106)
Did you include BTRFS in here just to mentioned all weak points of BTRFS and act like they also true for ZFS? Because that's not true at all.

> Maybe because in a virtualised environment, you loose many of the benefits of ZFS or BTRFS (they like to see the raw disk, not some abstract notation).

ZVOL exist...You can snapshot, close, send/recv it just like dataset. You can fine-tune it like any other dataset. ZVOL works just fine for VMs.

> Maybe because ZFS and BTRFS still have their bugs, small and big (hello RAID5 in BTRFS ;-) ) and can be heavily dependent on particular implementations.

Everything has bugs. Again, you put BTRFS here just to drive your point like if it's try BTRFS then it's true for ZFS?

> Maybe because BTRFS needs babysitting[1] (don't know about ZFS, I suspect it might).

Same as above.

> Maybe because the combination of XFS/ext4 plus LVM is more than good enough for most people as that route still provides snapshots etc.

Very different kind of snapshots...

I've been using ZFS on my desktop and laptops for over a decade now. You gotta make really kick-ass FS for me to switch off ZFS even for desktop.

Re:OpenZFS Merged to FreeBSD by squarefoot on Tuesday August 25, 2020 @06:40AM (#24269900)
> In all honesty I can't understand why would anyone use anything other than zfs nowadays for important data.

In datacenters many probably do, however most home/soho NAS machines are severely limited wrt memory and CPU power, which could be a limit. My self assembled NAS uses a Atom board to keep power requirements low since it stays always on, and I can't complain about its performance, however that CPU doesn't support more than 4GB RAM which is considered the minimum to properly use ZFS (NAS4free). Many cheap commercial NAS boxes used in small offices are even more limited. I wonder if there are any technical reasons preventing ZFS to operate (or be adapted to) in low memory environments.

Re:OpenZFS Merged to FreeBSD by amarshall on Tuesday August 25, 2020 @08:25AM (#24270597)
> 4GB RAM which is considered the minimum to properly use ZFS

This is mostly a myth with origins in the high-memory requirements of ZFS deduplication (which few should use, anyway). Of course, more memory allows for more caching, but that’s true of any filesystem on a “modern” OS.

Re:OpenZFS Merged to FreeBSD by djsumdog on Tuesday August 25, 2020 @11:02AM (#24272022)
Yea I only have 4GB of ram on my little HP MicroServer I use as a NAS with ZFS. The only issues I seem to have is that ZFS encryption seems to use more CPU than luks encryption, but it's still not too bad (the AMD Turion(tm) II Neo on it doesn't have AES instructions).

I might upgrade it to something with a better processor because I do want to run de-duplication on my 14TB drive at some point.

Re:OpenZFS Merged to FreeBSD by Wowfunhappy on Tuesday August 25, 2020 @07:27AM (#24270211)
I have used ZFS with 1 GB of RAM. It worked fine, it just wasn’t able to cache stuff to memory.
Re:OpenZFS Merged to FreeBSD by linsomniac on Tuesday August 25, 2020 @10:58AM (#24271975)
I wonder if it would be useful to have a Optane for the memory-hungry portions of ZFS, particularly the dedup tables, especially to be able to run it on very small, efficient machines.

My work backup server has "only" 32GB of RAM which is nowhere near enough to do dedup, even though there's a ton of stuff that could be deduplicated. Looks like it needs at least 64GB RAM to dedup, and that machine is maxed out.

But at home I'm in a similar situation to the parent. I'd like to set up a NAS, but I use it mostly for backups long term storage and don't want a big machine sitting idle all the time. A tiny Atom with Optane for cache and DDT might be ideal, in my thinking.

But for most of the storage use Backblaze is probably the right answer.

Re:OpenZFS Merged to FreeBSD by Rafuino on Tuesday August 25, 2020 @01:06PM (#24273484)
Would the Optane in this case be used as an L2ARC to handle part or all of the dedupe table? I'm trying to brush up on my ZFS terminology at the moment...
Re:OpenZFS Merged to FreeBSD by linsomniac on Tuesday August 25, 2020 @02:08PM (#24274179)
Currently, AFAIK, the dedup table (DDT) has to reside entirely in RAM. The L2ARC is a disk-based cache of filesystem blocks (a pretty clever design, worth reading up on), but a very different thing from the DDT.

You could definitely put the L2ARC on Optane, but Optane is probably better suited to the ZIL (ZFS Intent Log, basically the journal so writes can be quickly acked) if you aren't using literal NVRAM.

Breaking the DDT out so it could exist on a Optane isn't currently available (again, AFAIK, but I recall hearing about some changes in that department, I don't recall specifics and may be misremembering).

I've basically never had a machine big enough to comfortably use dedup except on trivial loads. My backup boxes, where I'd really like to use it, have all fallen over when enabling dedup because of the RAM requirements.

Re:OpenZFS Merged to FreeBSD by fomine3 on Tuesday August 25, 2020 @08:44PM (#24277907)
ZFS 0.8 supports a feature called "Allocation Class" that makes DDT able to working on "special" storage device (aimed to use NVMe) rather than RAM. It also supports to put metadata and small blocks on special device.

https://forums.servethehome.com/index.php?threads/zfs-alloca...

Re:OpenZFS Merged to FreeBSD by znpy on Tuesday August 25, 2020 @01:02PM (#24273443)
I've been running zfs on a 2gb machine with little I/O (it mostly receives snapshots) and had no issue so far.

Generally speaking, I will go with lower specs machines until I run into some actual problem. Haven't so far.

Re:OpenZFS Merged to FreeBSD by curt15 on Tuesday August 25, 2020 @06:46AM (#24269950)
Google still uses ext4 for its servers, while Facebook uses mostly btrfs nowadays.
Re:OpenZFS Merged to FreeBSD by nix23 on Tuesday August 25, 2020 @06:53AM (#24269994)
Google has on top of ext4 his GoogleFS, in theory you could even go 'without' a underlying FS, a bit like BlueStore from Ceph.
Re:OpenZFS Merged to FreeBSD by dannypgh on Tuesday August 25, 2020 @09:51AM (#24271321)
This does not sound right to me.

If you're talking about GFS, that's been gone for a decade, and was not a filesystem as far as the kernel was concerned.

Re:OpenZFS Merged to FreeBSD by nix23 on Tuesday August 25, 2020 @10:00AM (#24271425)
That's why i brought up the Ceph example, but counts for databases etc too, actually the early databases had all their own fs/data-structure on the platter.
Re:OpenZFS Merged to FreeBSD by vaylian on Tuesday August 25, 2020 @05:42AM (#24269583)
Maybe my understanding of ZFS is wrong, but what about storage requirements? As far as I understand ZFS, you take snapshots of all versions of all files and that can quickly lead to a lot of space usage.

But maybe there is also an automatic cleanup tool that deletes old snapshots after some time?

Re:OpenZFS Merged to FreeBSD by magicalhippo on Tuesday August 25, 2020 @06:09AM (#24269744)
You take snapshots when you want to take snapshots, and a snapshot allows you to view the data as it was at the time the snapshot was taken. Not entirely unlike a git commit.

The snapshot refers to the storage blocks/records on disk, not files as such, an important distinction since ZFS can expose block storage (zvol) as well as a "regular" file system.

Since ZFS is copy-on-write, the only storage you pay for with a snapshot are the blocks that have changed since the snapshot was taken (plus a little overhead). Thus for data that does not change much, a snapshot is almost "free".

The blocks are reference counted. Once a snapshot is deleted, ZFS decreases the reference count of the blocks referenced by the snapshot. Any block with a refcount of zero is considered free and thus that space is reclaimed. This happens when the block has changed since the snapshot was taken and there were no other snapshots referencing that block.

ZFS itself has no automatic deletion of old snapshots AFAIK, but there are tools built around ZFS that allow for periodic snapshotting and cleanup.

Re:OpenZFS Merged to FreeBSD by rakoo on Tuesday August 25, 2020 @06:21AM (#24269815)
Side-topic: the way you described how blocks and snapshots work is _exactly_ how git works, with references and blobs: as long as a blob can be reached by a reference (branch or tag), it will not be garbage-collected.

Turns out a DAG, as is used in both ZFS and git, is a good data structure in many use cases.

Re:OpenZFS Merged to FreeBSD by gruturo on Tuesday August 25, 2020 @07:46AM (#24270358)
Snapshots are something you control - they don't happen by themselves.

Furthermore, you only "pay" their storage cost for data which actually changes. On a 2TB volume, last snapshotted 1 week ago, if only 5GB have changed since then, that's the only storage overhead (ignoring for simplicity the folder structure itself).

Also, if a single data block changes 20 times since its last snapshot, you still only "pay" storage costs for its current version + the one sitting in the last snapshot.

Re:OpenZFS Merged to FreeBSD by chungy on Tuesday August 25, 2020 @12:35PM (#24273143)
It's unfair you've been downvoted. This is an excellent question and the responses were likewise excellent.

We need to encourage more of this :)

Re:OpenZFS Merged to FreeBSD by fomine3 on Tuesday August 25, 2020 @08:46PM (#24277917)
sanoid is popular tool for homelab user to scheduling snapshot creation/deletion.
Re:OpenZFS Merged to FreeBSD by whereistimbo on Tuesday August 25, 2020 @03:48AM (#24269018)
Hats off to anyone who involved there. Amazing achievement indeed.

Edit: change heads up to hats off

Re:OpenZFS Merged to FreeBSD by nucleardog on Tuesday August 25, 2020 @05:04AM (#24269422)
You probably mean “hats off to anyone involved” — generally meaning to give praise. “Heads up” would give the sentence more a meaning of “warning to anyone involved”.

Apologies if I’m being presumptuous or if this is just a late night typo. My wife speaks English as a second language so I know how confusing some of these idioms can be.

Re:OpenZFS Merged to FreeBSD by whereistimbo on Tuesday August 25, 2020 @05:30AM (#24269523)
Thank you for the correction. I'm not really sure about that but I type it anyway, hoping for correction from someone.
Re:OpenZFS Merged to FreeBSD by eitland on Tuesday August 25, 2020 @10:22AM (#24271617)
I had to look up the "heads up" phrase a few years ago and IIRC it is a warning from baseball.
Re:OpenZFS Merged to FreeBSD by Crontab on Tuesday August 25, 2020 @07:34AM (#24270264)
I would like to see NetBSD follow suite but I don't know if they have the manpower to do so.
Re:OpenZFS Merged to FreeBSD by y04nn on Tuesday August 25, 2020 @04:27AM (#24269238)
FreeBSD was staring to lack features compared to ZFS on Linux (ZOL), for example I'm not able to read encrypted datasets on FreeBSD but I'm able to do it on MacOS and Linux. More interoperability is very welcome.
Re:OpenZFS Merged to FreeBSD by rdc12 on Tuesday August 25, 2020 @08:13AM (#24270523)
From what I have heard, the amount of work needed in porting native encryption to the original FreeBSD port of ZFS was actually what led to this project.
Re:OpenZFS Merged to FreeBSD by nix23 on Tuesday August 25, 2020 @06:37AM (#24269887)
You can, just compile sysutils/openzfs
Re:OpenZFS Merged to FreeBSD by cryptonector on Tuesday August 25, 2020 @01:49AM (#24268431)
Two?? Three. More!
Re:OpenZFS Merged to FreeBSD by hestefisk on Tuesday August 25, 2020 @08:39AM (#24270684)
FreeBSD used to need a kernel module called opensolaris.ko in order to load ZFS (not sure if that’s still the case). So whilst ZFS has been ported to many OSes, it still relies / relied on a paravirtualized Solaris kernel. Note: it’s not a critique; I love ZFS and use it to secure all my data.
Re:OpenZFS Merged to FreeBSD by markjdb on Tuesday August 25, 2020 @09:41AM (#24271211)
"paravirtualized Solaris kernel" isn't really accurate - it's a collection of kernel interface shims. The whole thing is quite small, about 4KLOC on FreeBSD.