> So POSIX support is a given on most systems. Why replace it with something that breaks existing code?

You're not necessarily breaking existing code. Both macOS and Windows are built on non-POSIX primitives that have POSIX compatibility layers.

It seems that the conclusion most of industry has reached is that, whether or not POSIX is a useful API for your average piece of software, there are still better base-layer semantics to architect your kernel, IPC mechanisms, etc. in terms of than the POSIX ones. You can always support a POSIX "flavor" or "branded zone" or "compatibility subsystem" or whatever you want to call it, to run other people's code, after you've written all your code against the nicer set of primitives.

An potentially-enlightening analogy: POSIX is like OpenGL. Why do people want Vulkan if OpenGL exists? Well, because Vulkan is a more flexible base-layer with better semantics for high-efficiency use-cases. And if you start with Vulkan, the OpenGL APIs can still be implemented (efficiently!) in terms of them; whereas if you start with an OpenGL-based graphics driver, you can't "get to" (efficient) Vulkan support from there.

All that aside, though, I would expect that the real argument is: Fuchsia is for ChromeOS. Google are happy to be the sole maintainers of ChromeOS's kernel and all of its system services, so why not rewrite them all to take advantage of better system-primitive semantics? And Google doesn't have to worry about what apps can run on a Fuchsia-based ChromeOS, because the two ways apps currently run on ChromeOS are "as web-apps in Chrome", or "as Linux ELF executables inside a Linux ABI (or now also Android ABI) sandbox." There is no "ChromeOS software" that needs to be ported to Fuchsia, other than Chrome itself, and the container daemon.

Total speculation: but I seriously doubt that Fuchsia is specifically for chromeOS. The whole point of decent, efficient, simple, non-bug-prone APIs is that you probably want to implement pretty much everything on it. Simplicity and low-overhead allow for generality and flexibility.

If all you wanted to do was support chromeOS - well, typically you can add hacks even to a messy codebase to support specific usecases. And there are a bunch of linux and ?BSD distros that demonstrate that you can adapt such a system to even very small devices; small enough that there's not much niche left below. Moore's Law/Denard scaling may be comatose on the high-end; but lot's of long-tail stuff is generations behind; which implies that even really low-power IoT stuff that linux is currently ill-suited for will likely be able to run linux without too many tradeoffs. I mean; the original raspberry pi was a 65nm chip@700MHz - that's clearly overkill; and even if chip development never has a breakthrough again, there's clearly a lot of room for those kind of devices to catch up, and a lot of "spare silicon" even in really tiny stuff once you get to small process nodes.

But "being able to run linux" doesn't mean it'll be ideal or easy. And efficiency may not be the only issue; security; cost; reliable low latency... there are a whole bunch of things where improvements may be possible.

I'm guessing Fuchsia is going to be worse than linux for ChromeOS - in the sense that if ChromeOS really was what google wants it for, they could have gotten better results with linux than they'll be able to get with Fuchsia in the next few years and at a fraction of the cost. Linux just isn't that bad; and a whole new OS including all the interop and user-space and re-education pain is a huge price to pay. But the thing is: if they take that route they may end up with a well tuned linux, but that's it.

So my bet is that you'd only ever invest in something like Fuchsia if you're in it for the long run. They're not doing this "for" ChromeOS, even if that may be the first high-profile usage. They're doing this to be enable future savings and quality increases for use cases they probably don't even know they have, yet. In essence: it's a gamble that might pay off in the long run, with some applicability in the medium term - but the medium term alone just doesn't warrant the investment (and risk).

I guess I left a bit too much implicit about my prediction on what Google's going to do: I have a strong suspicion that Google sees the Linux/POSIX basis of Android as an albatross around its neck. And ChromeOS—with its near-perfect app isolation from the underlying OS—seems to be a way of getting free of that.

ChromeOS has already gained the ability to run containerized Android apps; and is expecting to begin allowing developers to publish such containerized Android apps to the Chrome Web Store as ChromeOS apps. This means that Android apps will continue to run on ChromeOS, without depending on any of the architectural details of ChromeOS. Android-apps-on-Android prevent Android from getting away from legacy decisions (like being Linux-based); Android-apps-on-ChromeOS have no such effect.

I suspect that in the near term, you'll see Google introducing a Chrome Web Store for Android, allowing these containerized, CWS-packaged Android apps to be run on Android itself; and then, soon after that, deprecating the Play Store altogether in favor of the Chrome Web Store. At that point, all Android apps will actually "be" ChromeOS apps. Just, ones that contain Android object files.

At that point, Google can take a Fuchsia-based ChromeOS and put it on the more powerful mobile devices as "the new Android", where the Android apps will run through Linux ABI translation. But in this new Android (i.e. rebranded ChromeOS), you'll now also have the rest of the Chrome Web Store of apps available.

Google will, along with the "new Android", introduce a new "Android Native SDK" that uses the semantics of Fuchsia. Google will also build a Fuchsia ABI layer for Linux—to serve as a simulator for development, yes, but more importantly to allow people to install these new Fuchsia-SDK-based apps to run on their older Android devices. They'll run... if slowly.

Then, Google will wait a phone generation or two. Let the old Android devices rot away. Let people get mad as the apps written for the new SDK make their phones seem slow.

And then, after people are fed up, they'll just deprecate the old Android ABI on the Chrome Web Store, and require that all new (native) apps published to the CWS have to use the Fuchsia-based SDK.

And, two years after that, it'll begin to make sense again to run "the new Android" on low-end mobile devices, since now all the native apps in the CWS will be optimized for Fuchsia, which will—presumably—have better performance than native Android apps had on Android.

From a branding perspective, that would be terrible. They've already invested a bunch in Google Play brand that isn't Android Apps (Play Music, Play Books, etc).

Seems more likely they'll allow HTML apps into the Play Store, eventually getting rid of the Web Store entirely. They've already done the WebAPK stuff to glue HTML apps into Android.

Google IO schedule has just been published.

Ironically they have two sessions named "The future of the Android app model and distribution on Google Play".

If, as I suspect, they'd be willing to rename ChromeOS to be "just what Android is now" (like how Mac OS9 was succeeded by NeXTStep branded as Mac OSX), then I don't see why they wouldn't also be willing to rebrand the Chrome Web Store as "what the Google Play Store is now." Of course, they'd keep the music, books, etc.; those are just associated by name, not by backend or by team.

But they wouldn't keep the current content of the Play (Software) Store. The fact that every Android store—even including Google's own—are festering pits of malware and phishing attempts, is a sore spot for Google. And, given their "automated analysis first; hiring human analysts never (or only when legally mandated)" service scaling philosophy, they can't exactly fix it with manual curation. But they would dearly love to fix it.

Resetting the Android software catalogue entirely, with a new generation of "apps" consisting of only web-apps and much-more-heavily-containerized native apps (that can no longer do nearly the number of things to the OS that old native apps can do!) allows Google to move toward a more iOS-App-Store-like level of "preventing users from hurting themselves" without much effort on their part, and without the backlash they'd receive if they did so as an end unto itself. (Contrast: the backlash when Microsoft tried that in Windows 8 with an app store containing only Metro apps.)

I expect that the user experience would be that, on Fuchsia-based devices, you'd have to either click into a "More..." link in the CWS-branded-as-Play-Store, or even turn on some setting, to get access to the "legacy" Play Store, once they deprecate it. It'd still be there—goodness knows people would still need certain abandonware things from it, and be mad if it was just gone entirely; and it'd always need to stick around to serve the devices stuck on "old Android"—but it'd be rather out-of-the-way, with the New apps (of which old Chrome Apps from the CWS would likely be considered just as "new" as newly-published Fuchsia apps upon the store's launch) made front and centre.

> Seems more likely they'll allow HTML apps into the Play Store, eventually getting rid of the Web Store entirely.

I would agree if this was Apple we were talking about (who is of a "native apps uber alles" bent) but this is Google. Google want everyone to be making web-apps rather than native apps, because Google can (with enough cleverness repurposed from Chrome's renderer) spider and analyze web-apps, in a way it can't spider and analyze native apps. Android native apps are to Google as those "home-screen HTML5 bookmark apps" are to Apple: something they wish they could take back, because it really doesn't fit their modern business model.

> The fact that every Android store—even including Google's own—are festering pits of malware and phishing attempts, is a sore spot for Google.

Lol, citation needed.

And then they will stop releasing Fuchsia's (and Android's) source code and become the new Microsoft of the 90s.

Take for example their "replace Intel ME with Linux" project

Agree 100%. I very much doubt it is to replace ChromeOS.

They may not replace ChromeOS, but I'm going to guess they'll replace the Linux kernel with Zircon.

Completely agree. I very much doubt Fuchsia is to replace ChromeOS.

> You're not necessarily breaking existing code. Both macOS and Windows are built on non-POSIX primitives that have POSIX compatibility layers.

I was under the impression that MacOS was built from a POSIX kernel, all the way down.

Nope; the XNU (macOS) kernel has Mach semantics (https://developer.apple.com/library/content/documentation/Da...), not POSIX semantics.

XNU does embed BSD (and so POSIX) semantics into the kernel—some of which are their own efficient primitives, since there's no way to efficiently implement them in terms of Mach. But whatever BSD syscalls can be implemented kernel-side in terms of Mach primitives, are.

It has both. Mach system calls are negative, BSD system calls are positive. The BSD side has system calls for stuff like fork() that would otherwise be pretty clearly in Mach's domain.

Hard to imagine going to be used for ChromeOS before Android. Android runs native on Chromebooks because it shares a common kernel with Android so it can run in a container.

That would be lost which is a huge deal. The new gnu/Linux on ChromeOS would be fine as it runs on a VM and would still work.

Now they could move Android to using a VM but that it less efficient and most importantly takes more RAM and CBs do not normally have a ton of RAM.

Linux kernel is not directly exposed to userspace Android apps, so it is actually irrelevant.

In fact, starting with Android 7 they have started to lock down NDK apps that try to use APIs not listed as stable.

It is relevant as unless they replace both ChromeOS and Android kernels you lose the ability to run Android native on a ChromeOS box.

With the introduction of Project Treble wouldn't a kernel swap be relatively easy providing the Treble interface contracts are met?

It would have to be as efficient as Linux as the big plus with ChromeOS is the ability to have a peppy computer on minimal hardware.

I am not convinced a micro kernel would be able to achieve.

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code?

Because POSIX has horrible security properties, and does not provide enough guarantees to create truly robust software. See for instance, the recent article on how you simply cannot implement atomic file operations in POSIX -- sqlite has to jump through 1,000 hoops to get something pretty robust, but it shouldn't be this way.

The biggest problem is the blocking IO. With async throughout you can design a proper concurrent system.

The Linux kernel has Async IO. For a while now.

You don't need async for a proper concurrent system. Systems have been concurrent before async IO. The trick is that when a process does an IO you yield it and put it on a wait list and run literally anything else until the kernel receives the OK from the hardware control and resumes the process.

Using Async IO in Linux merely means your specific thread won't be immediately suspended until you get data (or it's in a reasonably close cache)

It would be quite silly if Linux would wait for every IO synchronously, any single core system would immediately grind to a halt.

  > Linux kernel has Async IO

Linux offers some async io features, but does not offer async throughout.

In a fully async platform you would be able to do general-purpose programming without ever needing to use multithreading.

Example of a situation you can't do in linux: have a program doing select(2) or equivalent on both keyboard input and network input, in a single thread.

Since linux does not support this, you are steered to adapt solutions that are more complicated than a pure async model would be,

* Spin constantly looking for activity. These heats up your computer and uses battery.

* Have short timeouts on epoll, and then glance for keyboard input. This leads to jerky IO.

* Have child processes block on these operations and use unix domain sockets to feed back to a multiplexor (fiddly, kernel contention).

* The child-process thing but with shmem (fiddly)

* Something equivalent to the child process thing, but with multiple threads in a single process. (fiddly)

You would think that x-windows might help out here. What if you had a socket to X, and then multiplexed on that, instead of looking for keyboard input from a terminal? This opens new issues: what if X has only written half of an event to your socket when select notifies you? Will your X library handle this without crashing?

Rurban's comment above is correct. Linux is not async throughout.

On OSs that offer kevent you can get a fair bit further, but (I believe) you still can't do file creation/deletion asynchronously.

This is broken. (Woke from a sleep, face-palmed. I have been in Windows IPC land and got my wires crossed, sorry.) In linux you can select on both the stdin fd and network sockets in a single call. There is a way to get a fd for AIO also. AFAIK the sync-only file creation/deletion stands.

Linux is async throughout since it can do task switching on blocking.

You're not the only task running.

Or do you expect Linux to pause the system when you read a file until the DMA is answered?

Linux itself is fully async, anything blocking (even interrupts to some extend) will be scheduled to resume when the reason it blocked is gone or alternatively check back regularly to resume.

A program running on Linux can do a lot of things async as mentioned via POSIX AIO. It's not impossible, go seems to do fine on that front too (goroutines are put to sleep when doing blocking syscalls unless you do RawSyscall).

The conclusion that lack of 100% asynchronity means it can't be properly concurrent is also wrong. As evident that sending something to disk also doesn't not halt the kernel.

Which isn't part of POSIX.

AIO is definitely part of POSIX.

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/aio....

I thought you were speaking about some Linux specific APIs.

There are some Linux specific APIs that build on this and I've got the most experience with Linux so I was refering to that.

But Async IO is part of POSIX.

I was mostly responding to the comment which cited synchronous IO as a problem with POSIX. Linux is most widely used so on top of being my primary domain on the issue, it's going to ensure best understanding.

That's not to say it's implemented as efficiently as it could be.

See here: https://linux.die.net/man/7/aio

AFAIK, system-specific APIs such as epoll or kqueue are the way to go if you're serious about performing asynchronous IO?

Is there an example of an os which doesn't have blocking io? To me it seems that blocking io will be needed at some level. You can of course put wrappers around it and present it as async. But in many applications you want to go as low as possible, and in my imagination blocking calls will be the lowest.

It's the other way around. Low level is all async everywhere, blocking is sort of just telling the kernel to wait for async op to complete before running the process.

Midora, Singularity in an OS. Pony in a language.

The goal is to avoid blocking, not to offer async also. That's a lost cause.

Fuchsia once had the goal to offer async only, but then some manager decided that he needs some blocking, and then it was gone.

non-blocking only is lower level than blocking. You can always wait indefinitely for that callback, but usually you have a default timeout. L4 for example offers that API.

And yet here I am, working day in and day out on a POSIX system, uptime in the years range... strange.

Anything can be made to work with enough effort! Imagine how much easier it would be if POSIX actually had a better API.

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code?

POSIX is a lowest common denominator API, its under specified and loosely interpreted. Which follows from its initial goals, which was basically to specify the bits common between various UNIX implementations. Those implementations obviously didn't want to change their implementation to match a rigid standard, so a lot of wiggle room exists in the standard.

The end result is that is pretty much useless for anything beyond "hello world" kinds of applications, both in terms of portability, as well as actual behavior (I could list a lot of cases, but lets leave that to google, with the starting idea to look at a couple posix API's, say close()'s errno's and the differing cases and what causes them on different OS's/filesystems). That is why there isn't a single OS out there that is _ONLY_ POSIX compliant. You need look no further than the 15 year old https://personal.opengroup.org/~ajosey/tr28-07-2003.txt and consider that the gap has widened as more performance or security oriented core API's have been introduced and linux's POSIX layer is further refined upon them.

Plus, the core API's in no way reflect the hard reality of modern hardware, leaving the standard even more under specified in the case of threads and async IO, which have been poorly bolted on.

Then there are all the bits everyone ignores, like the bits about the posix shell (ksh), and how certain utilities behave, while completely ignoring important things like determining metadata about the hardware one is running on.

I leave you with: https://stackoverflow.com/questions/2693948/how-do-i-retriev...

and https://stackoverflow.com/questions/150355/programmatically-...

Which is pretty basic information about a modern machine.

> POSIX is a lowest common denominator API,...

Only useful for CLI tty old style apps and daemons.

> So POSIX support is a given on most systems. Why replace it with something that breaks existing code?

POSIX says that usernames and uids can overlap, and if a program takes either, it should accept the other a well. And if something could be a username or uid, it should be presumed to be a username, and resolved to a UID.

Now assume you have a user "1000" with uid 2000, and a user "2000" with uid 1000... you can see where this goes.

And this is why the BSDs have broken POSIX compatibility and require uids to be prefixed with # when they’re used on the CLI.

No, it does not say that. It says, specifically for certain standard utility programs such as chown and newgrp, that particular program arguments should always be considered to be user names if such user names exist even if they happen to be in the forms of numbers.

Nor is what you claim about the BSDs true. Aside from the fact that using the shell's comment character would be markedly inconvenient, especially since the colon is a far better choice for such a marker (c.f. Gerrit Pape's chpst), the OpenBSD and FreeBSD implementations of chown and newgrp do not have such a marker either documented or implemented, and in fact operate as the SUS says.

User names and user IDs are, by contrast, explicitly called out by the standard as strings and integers, two quite different things which have no overlap.

* http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...

* http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...

> But some bloat is the patina that all succesful systems take on over time.

What a nice analogy.

One man’s “bloat” is another man’s feature.

> But a lot of engineering effort went into making the core really nice.

I quite like Linux but I would not say the current state of the core kernel is "really" nice. Somewhat nice, maybe, but "really" nice? You just have to look around to see exemples of non-nice things (too much things in some headers, jumping between source files back and forth for no reason, excessive use of hand coded vtables, internal "framework" that are not as simple as they should, at other time lack of abstractions and excessive access to internals of structures, etc). Granted, it is way less buggy than some other softwares, but I think I'll know when I read a really nice core, and for now I've never had the feeling that Linux has one.

Still find it quite good, to be clear.

FreeBSD?

Looks like some parts of POSIX is implemented. See this page for their rationale: https://fuchsia.googlesource.com/docs/+/master/the-book/libc...

>Why replace it with something that breaks existing code?

https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02...