News for hackers. Stuff that matters.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @05:41AM (#21256912)
Somewhat related, I recently watched:

"By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives" https://www.youtube.com/watch?v=HrUvjocWopI

Slides: https://callfortesting.org/log/vBSDcon-2019-Dexter-BytheNumb...

And was surprised that zfs for windows (and hyper-v core) appear to be alive and comming along.

Anyone using zfs on windows? Are they also going off of zfsonlinux - and if so how's support for encryption comming?

Finally having shared, read write, encrypted home that works on both Linux and windows safely would be great!

Re:A detailed look at Ubuntu’s new experimental ZFS installer by jiggawatts on Tuesday October 15, 2019 @06:40AM (#21257197)
The ZFS-equivalent on Windows is meant to be ReFS, which is roughly equivalent to ZFS when layered on top of Storage Spaces.

I'm really keen to give ReFS+SS a go, but I keep hearing horror stories of abysmal parity-spaces performance, which can only be explained by poor coding and testing.

Even if ZFS has better performance, it's never going to have full support, and it wasn't written for the NT kernel IO model, so I'm dubious about it ever working well...

Re:A detailed look at Ubuntu’s new experimental ZFS installer by p_l on Tuesday October 15, 2019 @07:11AM (#21257344)
There is less mismatch between ZFS and WinNT IO model than between ZFS and Linux VFS.

Sincerely, someone who had to port kernel level code from Linux to NT.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by fuzzy2 on Tuesday October 15, 2019 @08:13AM (#21257617)
I once tried ReFS, back on Windows 8.1 IIRC. Everything was going fine until suddenly, in the middle of the day, Windows decided that the ReFS volume was corrupt. And then you discover there’s virtually no rescue tools. It nearly gave me a heart attack. After rebooting three times it decided everything was okay again.

I quickly moved away my data and never used ReFS since.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by xenospn on Tuesday October 15, 2019 @08:44PM (#21265849)
That happened to me with a beta version of APFS. I never got my data back.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @09:15AM (#21257994)
This entirely misses the fact that zfs (open zfs/zfsonlinux) is open and cross platform.

Which is almost more interesting than all the good stuff that comes with zfs (volume management, disk management, software raid-like functionality, encryption, snapshots, send/receive).

Its the first time we truly have a half-modern filesystem with safe write support and sane performance across many different os's. Even if just made dual booting freebsd and Linux this would be a fmgreat win imnho.

If the choice is between zfs and fat32...

Re:A detailed look at Ubuntu’s new experimental ZFS installer by pm7 on Tuesday October 15, 2019 @09:58AM (#21258377)
Unfortunately, FreeBSD uses different ZFS then Linux. Yes, it's crazy, but currently I think it is not guaranteed to be able to mount ZFS between these systems. Fortunately, FreeBSD is supposed to base their ZFS on Linux version in future.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by p_l on Tuesday October 15, 2019 @11:52AM (#21259770)
ZFS is mountable between Linux and FreeBSD so long as optional features are configured in an appropriate way - this is a benefit of OpenZFS "version 5000" scheme.

IIRC, one of the issues involves the layout of extended attributes. FreeBSD keeps to Solaris layout (which is essentially the same in concept to NTFS' one), while Linux for speeding up certain stuff (and because Linux can't use native xattrs to full extent anyway, due to using Irix API which is limited to PAGESIZE xattrs) has an optional feature to pack xattrs more densely.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @06:39AM (#21257190)
I used zfs on Linux with encrypted / for a while on Debian. It works really well, but unfortunately when I upgraded from Jessie to Buster, it did not recognize how I set it up and broke the system. I ended up having to reinstall.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @12:21PM (#21260210)
BTW, did you do a traditional in-place upgrade via apt? I've once or twice managed to break boot between Debian releases.. But not since lilo/early grub days.

Sounds like you probably ran into zfs not being in Debian proper - and buster unlocking dmcrypt, but not being able to mount zfs? Should normally be salvageable via booting previous Jessie kernel and initial ram/root disk (initrd).

At any rate, I gather you've moved on - just sound like an unlucky break - even with root on unsupported zfs.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @12:41PM (#21260477)
Yeah I did an in-place upgrade via apt. I for some reason thought zfs was in debian proper (contrib vs. main). Like I said in another post, I may try doing that again.

But then again, the main reason I haven't is because all of my stuff is backed up via Nextcloud, so reinstalling Debian is pretty quick/painless for me.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @12:16PM (#21260141)
Related: I moved to a new laptop at work with a zfs send over ssh of /home/myuser - and a mount+manual copy of what I wanted. Very straight forward - but not that different from an rsync to /home/myuser/olfstuff.

Zfs encryption was still a bit new, so both boxes uses dmcrypt on full disk plus efi plus /boot (technically redundant, but it works).

Looking forward to hopefully root on encrypted zfs in next Ubuntu lts.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by oarsinsync on Tuesday October 15, 2019 @07:29AM (#21257417)
Did you take a snapshot before performing the upgrade, and did you try to restore the previous snapshot before the reinstall?

Curious if snapshots can protect you even as far as OS upgrades gone wrong. I'd assume they can, if all filesystems are ZFS + snapshotted.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @10:16AM (#21258567)
To be honest I didn't feel like tinkering with it, I just reinstalled. Right now it is on ext4.

My issue was that for some reason Linux no longer could find the encrypted partition to decrypt, so I couldn't get it up to a point where I could even work with it.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by oarsinsync on Tuesday October 15, 2019 @10:25AM (#21258703)
Ah that's a shame. If you'd had a bootable USB stick or some pre-boot environment to use, you probably could have rolled back from a snapshot. At least, that's my understanding of the theory.

Your experience has definitely not discouraged me from wanting to ZFS my root partitions though. If only to provide the potential / theoretical ability to roll back a snapshot if necessary!

Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @10:41AM (#21258894)
To be honest, it was laziness on my part, I could have probably figured it out and fixed it. The snapshot ability was incredibly nice, don't get me wrong. I really liked it on my system. I may switch back in the near future.

I can post my "script" when I get home if you want to try it. I use script very liberally, it was my notes of the commands I used in order in order to get ZFS/dmcrypt working.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by ohthehugemanate on Tuesday October 15, 2019 @08:33AM (#21257712)
I've used (conceptually equivalent) btrfs snapshots to let me install arch and gentoo, always easily rolling back to my own heavily customized Ubuntu system. I've never tried it, but given a copy of the snapshot tools, you should even be able to rollback from formatting your root partition.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by pm7 on Tuesday October 15, 2019 @10:03AM (#21258424)
I think there are even some tools to do this directly in GRUB, at least for btrfs. We can have automatically created list of available snapshots.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @09:24AM (#21258070)
Hm, you used zfsonlinux 0.8 under Jessie? Or was this prior to general release of native encryption support?
Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @10:42AM (#21258912)
What do you mean by "native" encryption? I used dmcrypt+LUKS. That is file system agonistic.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @12:03PM (#21259952)
That's not zfs encryption, that's "just" encryption (ed: with zfs on top).

(open) Zfs now has native encryption:

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0

(ed: there are some potential benefits to zfs encryption - it's as cross platform as zfs (encrypt an USB drive, use on Linux, os x, windows, freebsd).

Zfs send/receive - receiver can take your encrypted backup - key doesn't have to travel.

Mount drives across dual boot - share encrypted /home between dual boot windows and Linux?

Not sure - but I think volume encryption is supported too. Might simplify encrypted swap (if you already "pay" for zfs). Note: there are apparently some issues with swap on zfs (volumes at least, possibly files too)

If you don't need suspend/resume - suppose you could share one encrypted swap volume across multiboot environments..) )

Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @12:13PM (#21260100)
Heh, ok neat! I did not use that then.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @12:22PM (#21260222)
If you did on Jessie I'd be more than a little impressed! :)
Re:A detailed look at Ubuntu’s new experimental ZFS installer by stiray on Tuesday October 15, 2019 @10:09AM (#21258490)
zfs import -af?
Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @10:43AM (#21258933)
Like I said elsewhere, I could have probably fixed it, but I was lazy. I also wanted to try out Wayland, but I ended up going back to Xfce. Maybe when Xfce moves to Wayland I will try again.
Re:A detailed look at Ubuntu’s new experimental ZFS installer by aasasd on Tuesday October 15, 2019 @07:35AM (#21257449)
Isn't encryption supposed to be a separate layer from the filesystem? Or is the merger specific to Windows? Or, is this ZFS' attitude toward features, like with the volume manager?
Re:A detailed look at Ubuntu’s new experimental ZFS installer by cyphar on Tuesday October 15, 2019 @07:56AM (#21257547)
ZFS "native encryption" is an upstream feature. The argument for doing it in the filesystem rather than as a separate layer is that it allows you to have far more flexibility than classic FDE (encrypted send/recv, you can use different keys for each filesystem, the encryption applies to all data written by ZFS so you don't need to worry about forgetting to set up LUKS on a new drive added to your pool, etc.)
Re:A detailed look at Ubuntu’s new experimental ZFS installer by pm7 on Tuesday October 15, 2019 @10:02AM (#21258406)
> How many drives can you fit into a laptop?

Why?

One more addition to other replies: ZFS needs access to disks for self-healing (bit-rot detection and repair). It means that in the past if we wanted encryption, same date needed to be encrypted twice (one for every mirror).

Now with encryption in ZFS, date is encrypted before creating redundancy. We can even do a scrub without encryption key.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @09:22AM (#21258051)
In addition to what @cyphar mentions - zfs is "storage manager". Disk, volume and filesystem. Some of the power and data guarantees relate to this design decision.

Encrypted /home/username filesystem with user-owned keys, is one neat trick,for example.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by kop316 on Tuesday October 15, 2019 @10:52AM (#21259042)
That is how Linux and FreeBSD handle it. They both present a block layer device that you can then put ZFS on.

The ZFS specific encryption IIRC is owned by Oracle, and they do not open source it.

Re:A detailed look at Ubuntu’s new experimental ZFS installer by e12e on Tuesday October 15, 2019 @12:01PM (#21259923)
Oracle "closed" (forked) zfs "back" and added encryption. But encryption is (now) in openzfs (who's canonical upstream has gravitated to "zfsonlinux" which is just the open project that ended up having the major mind-share).

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.8.0