Deduplication requires an absolutely enormous amount of memory, unless you have a dataset that has a huge amount of duplication for some reason (maybe a company or school where a lot of people have their own network storage and put the same large files in?) it is usually never the right choice to enable dedup.

It is cheaper to buy more disk than the memory to enable dedup (somebody could run the numbers). ZFS ARC also likes lots of memory and you get good performance gains with more memory allocated to it.

In other words, turn on LZ4 compression everywhere and don’t think about it otherwise.

This is my understanding as well. These days we default to zstd, though.

Oh good, I have it off. I was pondering that feature years ago when I set it up. Couldn't remember where I landed.

Deduplication in ZFS can be pushed onto dedicated flash storage now instead of requiring gobs of RAM.

Great channel for side attacks btw.

Doesn't even need SSD's to open a side-channel: https://lore.kernel.org/linux-btrfs/CAKDzk=-HZardsLFH5c9HYre...

Without having physical access to my hardware, where every disk is FDE anyway?

Physical access not required. Dedupe side attack required timing measurement.

Oh got some hate there.. pissed off the zfs kids with a dose of reality.

Dedupe can be used to infer secret data. .. more news at 11

How to do this?

Data loss bugs in ZFS?? Wow.

My personal approach to filesystem evaluation is pretty much "How long has it been since I've seen someone talk about a data loss bug". This is my first time hearing one for ZFS. BTRFS seems to have at least one post a year. Ext4 has almost none, but maybe that's because people trust it and blame hardware when unsure what the issue is.

Comparing zfs/btrfs with ext4 is not entirely fair. They're in different domains when it comes to scope and features. To some extent, silent corruption in ext4 is expected but in zfs it's a bug.