Finally, it’s unlikely that implementing it in the OS would be faster because there’s a bunch of flash translation logic and garbage collection that has to happen - much faster if all of that happens on the other side of the PCIe bus with a dedicated coprocessor with a dedicated SPI directly to raw flash.
We could start with a nice implementation in the OS. Probably with a way for applications to opt in and out. And then optimize over time. Optimizations could include better integration with the way NVMe devices actually work, rather than the way they pretend to work to the rest of your computer.
Additionally, apps can do whatever they want in custom partitions if they need better storage perf (databases for example do this). Getting lower level than that is probably doable, but you run the risk of becoming firmware that is tied to particular drive controller design
Remember that computer storage technology is over 70 years old, many many things have been tried/implemented in that time.
Has this been tried? It seems like an obvious idea to me, but operating systems have evolved remarkably little in the last few decades. Especially the filesystem.
Well that seems a bit over the top. You would pay the price of full consistency for every write, while just one write every 3 years will fail. It would be insanely slow and wasteful.
Not to mention, that would basically disallow any kind of OS or hardware caching, which is immensely important for 99% of use cases.
> the OS should work with hard disk manufacturers to make sure their devices meet whatever expectation the OS makes of them
This is a bit of wishful thinking. We've seen already on numerous cases (Nvidia?) that OS developpers have very little leverage on hardware manufacturers. The best they could do is issue some kind of "Linux certified" label, which honestly nobody would care about.
Also, I guess there is some form of optimum that users and providers reach in terms of stability versus performance. If one OS provides x10 performance for 99.99% of uses while an other one provides 100% safety for /10 performance, I'm not sure users would choose the latter.
Why would it disallow caches? Just issue writes to a new location on disk and switch reads across to it after the write transaction has committed.
In any case, most writes aren’t performance sensitive. I’m thinking things like saving a file when code editing. My editor shouldn’t need to do some obscure, fragile renaming dance just to overwrite a file safely. It wouldn’t write 10x slower. Just have the OS do what it needs to do so my editor doesn’t have to.
And big transactions (like installing software or OS updates) could just do the whole update to a new location and then atomically swap to it. It shouldn’t cost any performance at all beyond a few pointers being moved around at the end of the transaction.
Maybe doing it for all writes is too much. Let databases opt out if they want to do their own thing. But I think it would be a much better default. Right now all good software does some version of this themselves. An OS level solution would allow all those hacks to be removed. If anything, I’d expect performance to improve.
You seem to be talking about _application level_ atomicity, like when you issue 4 write() to a file and want an "all or nothing" flush of these 4 writes.
I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk".
Am I understanding correctly?
> I'm talking about OS level atomicity, as in "I want a write() (or fsync) to block until it is flushed to disk".
Thats not atomicity. Thats a blocking API. And we already have an fsync that does that.
Blocking and atomicity are unrelated properties. You could have a blocking write thats not atomic (so if your computer turns off halfway through the write, it might have only half completed). And you can have an atomic write thats non-blocking (async). Or both or neither.
It's the same in this context. fsync uses a blocking API to implement atomicity. That is, after an fsync() all previous writes should have been written.
> we already have an fsync that does that.
Did you actually read the article...? The whole thing is about the fact that disk controllers can eagerly notify that a write was flushed to disk while in fact it is still in the disk write cache, and could have been either partially, fully or partly written, thus tricking fsync().
No, it doesn’t.
In that context fsync is a write barrier. It doesn’t make writes atomic. The difference is this: If you issue a handful of writes then your computer crashes, some, all or none of the writes may have completed. And some writes might be “skewed” - where part of the write succeeded and part did not. Atomicity means a set of actions happens as one indivisible unit, and it either all happens or all doesn’t happen. Think database transactions. Filesystem writes are not like that. They are not atomic operations[1].
Fsync does not change the behaviour of the write system call. Fsync only stalls your program until all writes have been written to disk, at least assuming your drive isn’t lying to you. Just like a memory barrier. If your computer crashes before fsync returns, all bets are off.
[1] With the exception of aligned writes under the device block size. Allegedly. I’d still like to see this actually documented somewhere.
It was recently deprecated because almost nobody uses it.
https://learn.microsoft.com/en-us/windows/win32/fileio/trans...
Non-atomic stuff is simpler on the implementation. Even if the behavior is worse for the user, it's easier to write, and you have fewer requirements to satisfy. So, not having a real pressure from the customers, it's just easier not to do this.
Very few apps need more than the ability to atomically swap a file.
And for the complicated multi write cases, a lot of software just use sqlite which deals with it.
No structural reason no, but a lot of other reasons. You can install your favourite atomic file operations library in your PL of choice and run some benchmarks to identify reason #1.
Personally I think it should be the responsibility of the operating system. All filesystem operations on modern computers should be atomic, and either entirely succeed or entirely fail. And the OS should work with hard disk manufacturers to make sure their devices meet whatever expectation the OS makes of them.
Hilariously, the OS already uses transactions for its own filesystem metadata operations. But they don't share that capability with end user applications, which end up either being quietly broken in the event of power loss (most apps) or horribly complex to make up for the OS's shortfalls (most databases).
There are dozens of solutions in userland, from git's content-addressing to on-disk B+trees. There should just be a simple, effective, correct transactional filesystem API provided by the OS that applications can simply use. The best part of this proposal: a solution implemented by the OS directly would probably be faster than what most people would implement in userland anyway. Its ridiculous this doesn't already exist.
There's simply no structural reason for power failure (or application crashes) to be able to put hard disk data into an inconsistent, corrupt state.