News for hackers. Stuff that matters.
Re:ZFS Root Filesystem on AWS by zorked on Thursday December 29, 2016 @02:38PM (#13279465)
Why?
Re:ZFS Root Filesystem on AWS by jlgaddis on Thursday December 29, 2016 @03:06PM (#13279703)
Ideally, ZFS has exclusive control over the storage. When it's virtualized, it doesn't and there may be various HBAs, raid controllers, etc., in between ZFS and the actual disks. These can (do) "get in the way" and you (can) lose one of the biggest features of ZFS (data integrity and error correction).
Re:ZFS Root Filesystem on AWS by lscotte on Thursday December 29, 2016 @03:26PM (#13279835)
The keyword is definitely "ideally" :-) With a cloud provider like AWS, storage is always virtualized - so we've always got that working against us. I see ZFS in AWS more about flexibility than data integrity, although having said that ZFS should do just as well or better (certainly no worse) than EXT4, XFS, or BTRFS for reliability. The ability to add storage dynamically without having to move bits around is powerful.
Re:ZFS Root Filesystem on AWS by drvdevd on Thursday December 29, 2016 @05:41PM (#13280831)
And I've actually had (a couple times now) random hardware failings, with random notification emails from Amazon, on EBS storage on AWS, with accompanying data loss.

Might as well treat your zpool like it's on real hardware and configure raidz accordingly. The cloud does have real, problematic hardware behind it, and it's important to remember that.

[edit] especially if you can configure your block devices such that you know they're sitting on different physical hardware at the cloud data center, you will gain that benefit of ZFS

Re:ZFS Root Filesystem on AWS by simcop2387 on Thursday December 29, 2016 @03:15PM (#13279766)
For a little more detail about this, it's somewhat common for these controllers to lie about when thinks hit the disk. They'll cache it and then write either out but if they lose power or a write gets corrupted zfs will have no way of discovering it until it's too late.
Re:ZFS Root Filesystem on AWS by bsuh on Thursday December 29, 2016 @03:25PM (#13279829)
Are other file systems better off with recovery when interacting with virtual disks and host power failure?
Re:ZFS Root Filesystem on AWS by IgorPartola on Thursday December 29, 2016 @03:53PM (#13280043)
They are not, but they are better at swallowing the errors and not bothering you with such details. ZFS fails fast & early, while EXT4 will fail when you realize your Postgres DB is borked.

I guess it's possible that some type of disk command timing could cause unexpected lockups or slowdowns that you wouldn't get with a system that doesn't try to control the hardware to quite the same extent as ZFS, but my (cursory) understanding is that it's rare/hardware specific.

My personal take is that running ZFS on hardware that lies is no worse than running EXT4 on it. YMMV as I'm not a storage expert.

Re:ZFS Root Filesystem on AWS by pkaye on Thursday December 29, 2016 @03:40PM (#13279940)
Search online for published papers related to "IRON File Systems." Some researchers injected errors into various parts of common file systems and see how well they recovered. I think ZFS was the best of the bunch though that research is from a few years back and things may have improved elsewhere.
Re:ZFS Root Filesystem on AWS by fulafel on Friday December 30, 2016 @01:05AM (#13282858)
The IRON paper doesn't mention ZFS, I found some citing papers that focus on ZFS but have no comparisons.
Re:ZFS Root Filesystem on AWS by pkaye on Friday December 30, 2016 @01:23AM (#13282912)
Yes you are right. I believe it was a different paper by the same research group... http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf
Re:ZFS Root Filesystem on AWS by fulafel on Friday December 30, 2016 @12:39AM (#13282764)
If the storage lies about syncs, the best you can hope for is replaying a consistent state somewhere in the past. Log structured filesystems with checksums would be a good bet here.
Re:ZFS Root Filesystem on AWS by ryao on Thursday December 29, 2016 @03:39PM (#13279936)
The short answer is no.
Re:ZFS Root Filesystem on AWS by aidenn0 on Thursday December 29, 2016 @11:35PM (#13282545)
Short answer "no" with a longer answer of "ZFS is sometimes better than most other options"
Re:ZFS Root Filesystem on AWS by deelowe on Thursday December 29, 2016 @04:22PM (#13280235)
There are virtualization solutions that provide exclusive access to storage. Hyper-V surprisingly is one such solution and I've been running ZFS inside of Hyper-V instances for a couple of years now with no issue.
Re:ZFS Root Filesystem on AWS by beachstartup on Thursday December 29, 2016 @04:46PM (#13280427)
pass-through device virtualization of various types provide this facility.
Re:ZFS Root Filesystem on AWS by agumonkey on Thursday December 29, 2016 @04:44PM (#13280406)
Somehow it's a good thing ZFS gets to be tried on AWS, it might trigger some bugs.
Re:ZFS Root Filesystem on AWS by scoot on Thursday December 29, 2016 @05:51PM (#13280904)
"there may be various HBAs, raid controllers, etc., in between ZFS and the actual disks."

Or in the case if EBS - NFS and a network stack. EBS is "Emulated Block Storage" despite what AWS may like you to think.

Re:ZFS Root Filesystem on AWS by cthalupa on Saturday December 31, 2016 @03:43AM (#13290192)
Source on EBS being NFS?
Re:ZFS Root Filesystem on AWS by Unklejoe on Thursday December 29, 2016 @09:37PM (#13282093)
Does this same idea apply when running ZFS on top of the Linux LUKS mapper (whatever the proper terminology is)?
Re:ZFS Root Filesystem on AWS by lscotte on Friday December 30, 2016 @11:52AM (#13285340)
I haven't tested it with /boot in ZFS, but I have a laptop that has an EXT /boot partition and a LUKS partition with ZFS for everything else (/, /home, etc). It works just fine - and I've never noticed any performance issues with this setup - obviously LUKS has some overhead, but running ZFS on LUKS works as well as EXT4 or BTRFS on LUKS, with all the advantages of ZFS.
Re:ZFS Root Filesystem on AWS by dekhn on Thursday December 29, 2016 @03:25PM (#13279830)
there's nothing ZFS specific about this. Your admins should be ensuring that you're not doing concurrent access to shared storage, or if you are, you're using a lock-based cluster manager.
Re:ZFS Root Filesystem on AWS by briankwest on Thursday December 29, 2016 @02:47PM (#13279528)
Looks like its not an issue, Was just reading the warnings a year or so ago specifically relating to FreeNAS and how it would corrupt your data.