But that approach will never win the hearts of UNIXy kernel devs, and I guess until Quantum computers take over we are past the point of having alternative OSes to UNIX based ones, because no one is going to spend the required money to do that.
So a migration path towards safety is required.
I think we're already half the way there, Amazon Linux is how widely deployed already? The surface area between application logic and OS code is shrinking by the year, once serverless goes mainstream then even application code will start to get rare, pushed to only those specific projects that require human interaction. Everything else is 'mere' computation, something that can be specified purely algorithmically and executed by anything.
The OS will only be relevant in the context of personal computing, the human interface to the machine kingdom which shed those trappings decades ago.
20-30 years down the road I see a translation layer unifying all three major, now solely personal, operating systems, and a fundamental, free software core being driven out, eventually supplanting proprietary development. Apple will do its best to retain tight coupling between their hardware and software, but they'll slowly fade just like Microsoft did as programming itself is revealed to consist of two main activities, intricate UX and comparatively dumb data plumbing, with a small priesthood of hardware driver maintainers / database developers for those who still want to do 'real' programming.
Everybody will be able to code at some level and it'll be amazing.
Something like vim has archaisms that are only mildly depended on, allowing a project like neovim to credibly call itself the future. But Python 2 still has a large install base of software that's still running it. Python 3 didn't offer a smooth enough upgrade path from Python 2, so we're seeing the slow bifurcation of the community into two language. This also happened with Rails 2, there is still large enough install base of Rails 2 software that there's a dedicated effort to support it, Rails LTS.
These bifurcations will last until gradual enough upgrade paths are made and failure modes are isolated enough so that moving a project off of old tech and onto new tech can be accomplished without business investment. It's merely another user story to be put in the pipeline. Old applications will slowly get either abandoned or migrated or reinvented over time.
This process fails when you consider underlying infrastructure, which by its nature largely has a much deeper base of software that depends on it. Applications can have many dependencies, but those dependencies tend to be well-enumerated, as not enumerating them leads to a maintainability nightmare for people unfamiliar with the system. And nothing but other business software depends on the application.
But infrastructure code's dependency structure is inverted. Many more applications depend on infrastructure and infrastructure tends to have comparatively fewer dependencies of its own. When infrastructure has lots of dependencies it makes it harder to scale, and scale is the whole point of infrastructure. This need for infrastructure to all 'hang together' so as to provide for scale is what's behind the modern concept of an operating system. Applications tend to rely first on the software stack, second on the operating system. It's much easier take a Rails application built to run on Ubuntu and migrate it so it can run on Red Hat or Docker than it is to get it to run on Sinatra, god forbid another general purpose programming language. In fact, it's so much easier that it's considered a major red flag for this to not be the case. Operating systems are more fungible than software stacks.
This process of bifurcation followed by slow abandonment of harder-to-maintain systems in favor of easier-to-maintain ones is just going to slowly reach its apotheosis when line of business software that is much of a career programmer's bread and butter is going to slowly lose all attachment to, not just the operating system, but also the software stack as well. FaaS, Function as a service, is something you can add to your application right now, and once you do that work once, it just gets easier and easier to offload portions of the application that simply glue things together or compute things, to a magical land where teams of elves do literally everything else for you for a small fee, is going to become mainstream sooner rather than later.
I see wasm as a step towards this inevitability.
Then we have JVM, MSIL and DEX.
What would wasm bring that isn't already done?
What we haven't had before are architecture-independent executables that can be compiled into from a large number of programming languages.
So how many languages do they need to be to be considered large?
Don't take me wrong, I love those bytecode environments, including Web Assembly.
I just don't think Web Assembly is that much different from other ones, other than politics.
Look for IBM redbooks on IBM i and IBM z language environments.
How was the performance of the old mainframe formats?
The kernel has a JIT and they are converted into native code at installation time, or when an operator decides to retrigger the operation.
In a way, Android (5 - 7), iOS bitcode and WP 8.x MDIL follow similar ideas.
And something that I forgot, MSIL also has support for C and C++, hence why all C# 7.x performance improvements don't require any runtime change, just exposing features that C and C++ already make use of on MSIL.
The mainframe ones, are just like any other OS, code gets generated at installation time, then they are OS processes just like any other.
Java and MSIL depend on how they get used.
Both have bytecode verifiers, and multiple ways to deploy.
Either as single runtime, where strong typing plays the role of sandboxing, or as multiple processes/containers.
Just to use .NET as example, not going to make an exhaustive list.
.NET Micro Framework deployed on Netduino, the runtime is the OS, only strong typing takes place.
.NET Framework on Windows, and .NET Core on Windows/Mac/Linux, each CLR instance gets their own OS process, which might be additionally wrapped into an OS container.
.NET Core on UWP, applications run as separate sandboxed process using the UWP sandbox model.
Xamarin on iOS runs with the iOS sandbox model, after AOT compiling for deployment.
Xamarin on Android, runs with the Android sandbox model where binary code gets dynamically loaded into ART instance, sandboxed via SELinux and Linux container mechanisms.
And regarding security, Web Assembly still fails short of preventing memory corruption inside of the loaded module.
I was hoping pointers would be tagged, but that doesn't seem to be part of the design goals.
> But that approach will never win the hearts of UNIXy kernel devs
I don't think this is fair. The Linux kernel contains (as of writing) 17913052 lines of C. If you have a plan to rewrite 18 million lines of code and patch every single dependency, fine. But until then such concerns will dominate.
It's also worth noting that the Rust many people write is not the low-level stuff that C programmers and systems programmers write. See https://www.cl.cam.ac.uk/~srk31/research/papers/kell17some-p... for a better explanation of that.
Not to rewrite C into something else, but to get out of the current escalation of kernel exploits, now that Linux is a commodity OS, it gets targeted as such
https://events.linuxfoundation.org/events/linux-security-sum...
Hence Kernel Self Preservation Project, removing VLAs from kernel code, adding all possible ways of mitigation, and hopping that eventually tagged pointers like SPARC and ARM have become common across all supported hardware.
The charts shown by Google employees regarding exploits are quite telling.
So there are only two options left, make C safe in some way no matter what, or just throw everything away.
The Linux Security Summit 2018 set of talks regarding kernel mitigation exploits is pretty clear how bad security exploits due to C are turning out to be.