Re:What's So Bad About Posix I/O?
by glennklockwood on Tuesday September 12, 2017 @10:55AM
(#15227767)
The context was definitely HPC, but that's mostly because it's the last place where this problem persists in a very big way. The WORM workloads that serve up most of the data we consume globally just rejected POSIX entirely and created the scalable object interfaces we know and love. It was very obvious that having every Roku in the world mount a Netflix file system over the WAN was insane, so nobody had to write an article calling for the end of that bad behavior.
Updating POSIX, whether it be through more expressive consistency options or just giving middleware developers the knobs they need to implement such mechanisms themselves, has come up before[1], as you say. My worry is that many of the serious efforts from years past were ahead of their time, and hyperscale has gone their own way so the ship has sailed on POSIX I/O.
This reminded me of another attempt to embed ordering in a filesystem in a usable and performant way, Featherstitch:
http://featherstitch.cs.ucla.edu/
I see the CCFS paper cites it to say that CCFS is better, but i haven't dug into why.
The author's on much firmer ground when he talks about POSIX consistency semantics. While we can't control misguided applications' (ab)use of locks, we can certainly offer models that don't require serializing things that the user never wanted or expected to be serialized, or making them synchronous likewise. As I've also written before[1], we need more ways to describe the user's desired level of ordering and durability, separately, giving the implementation maximum flexibility to optimize I/O while preserving the guarantees that users really care about. The setstream primitive in CCFS[2] is a good example of the sorts of things we need here. I'm not at all convinced that object-store-centric approaches are the way to go here, but some evolution is clearly needed.
[1] http://pl.atyp.us/2016-05-updating-posix.html section under "fsync"
[2] https://www.usenix.org/system/files/conference/fast17/fast17...