Frank Soltis' book is recommended reading: https://www.amazon.com/dp/1882419669/

It's a shame hardly anyone knows about it. Those things are a joy to use. You can get a free (limited, but still useful) AS/400 user account to play around with at http://pub400.com/. I really recommend it.

(Disclaimer: I'm slowly working on a system that resembles AS/400 in many ways, but optimized for analyzing and reporting on very large timeseries databases. It's intended for business applications that require a combination of scheduled reports and fast ad hoc analysis of big timeseries data, initially the oil & gas industry (which is where I work in my “real job”).)

Note that EROS is not providing a write-through cache. It's providing a write-back cache using checkpointing coupled with a journalling capability and ability to explicitly sync data.

So it's leaky: Your application needs to know that it needs to structure it's writes to memory so that they will make sense if the system comes back up with some of the data missing, and needs to know how to use the journalling functionality.

It can't just act as if it's running in RAM forever.

Check out http://wiki.c2.com/?TransparentPersistence

If you just pretend you're running in RAM, and the system crashes, you will lose the data between the crash point and the last checkpoint unless you have explicitly used the journalling mechanism. Often that is acceptable. E.g. since you're restoring the program at the same point in time, if the changes are entirely based on data that were in the system at the point of the last checkpoint, it will just redo the work to calculate the changes.

But if there are side-effects, that is often not going to be acceptable. E.g. database updates that the system has said were committed will suddenly disappear.

To solve that, EROS has a journalling mechanism to allow you to give guarantees about specific data that changes in between checkpoints, but that requires applications to explicitly use it to tell the OS what needs to be saved when so that the application can guarantee that a given piece of data has been durably recorded when it promises a client it has been recorded, and that the writes get correctly ordered.

That's a sensible compromise - if you do it right, it only needs to touch the "boundaries" where the system does IO.

> But if there are side-effects, that is often not going to be acceptable. E.g. database updates that the system has said were committed will suddenly disappear.

Yes, not every system will forever be recoverable. You can definetly crash at just the right moment to ruin your year. I'd still like the other safety constraints that this provides because I still think that even if N (where N is the number of threads available on the CPU) processes have a chance of being corrupted we're still going to be saving the rest of the processes on the system that aren't currently transacting with one another.

The point is more that it doesn't mean you can just treat things like we do RAM now. It ends up being closer to how you'd work in apps that use mmap'd files to back persistent data.

The capabilities model was interesting too.

I think my biggest problem with a persistence model like that, though, is that I suspect it would encourage not thinking seriously about state, and ideally I'd prefer a system where state is minimized. E.g. compare EROS with the virtual opposite: Android, for example, where apps might find themselves killed anytime. Some apps handle it poorly and go through lengthy initialisation processes again when restarted, but many maintain the illusion of being fully presistent to the extent that users can rarely tell if they've started from scratch or not.

I'd love to see more work into allowing the illusion of a persistent process with little developer effort, though. Perhaps OS-level per application checkpointing support that the application can have control over (allowing the app to control exactly what gets checkpointed, and when to ditch the checkpointed state to reinitialise). So "cleanup" can occur by restarting processes transparently for users while hopefully providing most of the benefits of persisting state.

Perhaps coupled with OS-level checkpointing of the information required to bring said "virtually-persistent" apps back up in the same state after a reboot.

https://en.wikipedia.org/wiki/MUMPS

Setting data in memory is the same as setting data on disk, the only difference is the name of the variable:

s X=1 ; store 1 in variable named X, in memory.

s ^X=X ; store 1 in variable named X, on disk.

s X=^X ; load disk to memory

These are all problems that have been solved on EROS based system. They used to do demos where they would setup a system and have someone start working on some code or a text document, they'd pull out the power plug of the system, plug it back in and the user would be right were they left off. No data loss, no corruption, just back to work.

None of that was handled in user space. That was all opaque and you didn't have to worry about it at all.

I'm guessing the system must effectively "checkpoint" your work regularly and sync to disk to avoid partially saving a state and corrupting the data.

This isn't terribly different from working on a memory mapped file except that it also saves the ephemeral state of the running program so it can be restored. But I still don't understand how it's not going to be horribly slow when you start your program and the first thing it does is allocate 4GB of memory for its workspace. Synching all of that data to disk is a massive undertaking, and this isn't an uncommon use case, people start virtual machines all of the time.

And paging is slow as balls. That's what this whole article is about, modern machines are unusable when they start paging.