https://unix.stackexchange.com/a/223763

With regards to POSIX, it is amazing how well this API is holding up. There are quite a few implementions from GNU, BSDs, Microsoft (at least partial support in MSVC) and a few others (e.g. musl). So POSIX support is a given on most systems. Why replace it with something that breaks existing code?

https://www.musl-libc.org/faq.html

Not to say there is no bloat. But some bloat is the patina that all succesful systems take on over time. Is the bloat small enough to be managed and/or contained? I say yes.

You're not necessarily breaking existing code. Both macOS and Windows are built on non-POSIX primitives that have POSIX compatibility layers.

It seems that the conclusion most of industry has reached is that, whether or not POSIX is a useful API for your average piece of software, there are still better base-layer semantics to architect your kernel, IPC mechanisms, etc. in terms of than the POSIX ones. You can always support a POSIX "flavor" or "branded zone" or "compatibility subsystem" or whatever you want to call it, to run other people's code, after you've written all your code against the nicer set of primitives.

An potentially-enlightening analogy: POSIX is like OpenGL. Why do people want Vulkan if OpenGL exists? Well, because Vulkan is a more flexible base-layer with better semantics for high-efficiency use-cases. And if you start with Vulkan, the OpenGL APIs can still be implemented (efficiently!) in terms of them; whereas if you start with an OpenGL-based graphics driver, you can't "get to" (efficient) Vulkan support from there.

All that aside, though, I would expect that the real argument is: Fuchsia is for ChromeOS. Google are happy to be the sole maintainers of ChromeOS's kernel and all of its system services, so why not rewrite them all to take advantage of better system-primitive semantics? And Google doesn't have to worry about what apps can run on a Fuchsia-based ChromeOS, because the two ways apps currently run on ChromeOS are "as web-apps in Chrome", or "as Linux ELF executables inside a Linux ABI (or now also Android ABI) sandbox." There is no "ChromeOS software" that needs to be ported to Fuchsia, other than Chrome itself, and the container daemon.

If all you wanted to do was support chromeOS - well, typically you can add hacks even to a messy codebase to support specific usecases. And there are a bunch of linux and ?BSD distros that demonstrate that you can adapt such a system to even very small devices; small enough that there's not much niche left below. Moore's Law/Denard scaling may be comatose on the high-end; but lot's of long-tail stuff is generations behind; which implies that even really low-power IoT stuff that linux is currently ill-suited for will likely be able to run linux without too many tradeoffs. I mean; the original raspberry pi was a 65nm chip@700MHz - that's clearly overkill; and even if chip development never has a breakthrough again, there's clearly a lot of room for those kind of devices to catch up, and a lot of "spare silicon" even in really tiny stuff once you get to small process nodes.

But "being able to run linux" doesn't mean it'll be ideal or easy. And efficiency may not be the only issue; security; cost; reliable low latency... there are a whole bunch of things where improvements may be possible.

I'm guessing Fuchsia is going to be worse than linux for ChromeOS - in the sense that if ChromeOS really was what google wants it for, they could have gotten better results with linux than they'll be able to get with Fuchsia in the next few years and at a fraction of the cost. Linux just isn't that bad; and a whole new OS including all the interop and user-space and re-education pain is a huge price to pay. But the thing is: if they take that route they may end up with a well tuned linux, but that's it.

So my bet is that you'd only ever invest in something like Fuchsia if you're in it for the long run. They're not doing this "for" ChromeOS, even if that may be the first high-profile usage. They're doing this to be enable future savings and quality increases for use cases they probably don't even know they have, yet. In essence: it's a gamble that might pay off in the long run, with some applicability in the medium term - but the medium term alone just doesn't warrant the investment (and risk).

ChromeOS has already gained the ability to run containerized Android apps; and is expecting to begin allowing developers to publish such containerized Android apps to the Chrome Web Store as ChromeOS apps. This means that Android apps will continue to run on ChromeOS, without depending on any of the architectural details of ChromeOS. Android-apps-on-Android prevent Android from getting away from legacy decisions (like being Linux-based); Android-apps-on-ChromeOS have no such effect.

I suspect that in the near term, you'll see Google introducing a Chrome Web Store for Android, allowing these containerized, CWS-packaged Android apps to be run on Android itself; and then, soon after that, deprecating the Play Store altogether in favor of the Chrome Web Store. At that point, all Android apps will actually "be" ChromeOS apps. Just, ones that contain Android object files.

At that point, Google can take a Fuchsia-based ChromeOS and put it on the more powerful mobile devices as "the new Android", where the Android apps will run through Linux ABI translation. But in this new Android (i.e. rebranded ChromeOS), you'll now also have the rest of the Chrome Web Store of apps available.

Google will, along with the "new Android", introduce a new "Android Native SDK" that uses the semantics of Fuchsia. Google will also build a Fuchsia ABI layer for Linux—to serve as a simulator for development, yes, but more importantly to allow people to install these new Fuchsia-SDK-based apps to run on their older Android devices. They'll run... if slowly.

Then, Google will wait a phone generation or two. Let the old Android devices rot away. Let people get mad as the apps written for the new SDK make their phones seem slow.

And then, after people are fed up, they'll just deprecate the old Android ABI on the Chrome Web Store, and require that all new (native) apps published to the CWS have to use the Fuchsia-based SDK.

And, two years after that, it'll begin to make sense again to run "the new Android" on low-end mobile devices, since now all the native apps in the CWS will be optimized for Fuchsia, which will—presumably—have better performance than native Android apps had on Android.

Seems more likely they'll allow HTML apps into the Play Store, eventually getting rid of the Web Store entirely. They've already done the WebAPK stuff to glue HTML apps into Android.

Ironically they have two sessions named "The future of the Android app model and distribution on Google Play".

But they wouldn't keep the current content of the Play (Software) Store. The fact that every Android store—even including Google's own—are festering pits of malware and phishing attempts, is a sore spot for Google. And, given their "automated analysis first; hiring human analysts never (or only when legally mandated)" service scaling philosophy, they can't exactly fix it with manual curation. But they would dearly love to fix it.

Resetting the Android software catalogue entirely, with a new generation of "apps" consisting of only web-apps and much-more-heavily-containerized native apps (that can no longer do nearly the number of things to the OS that old native apps can do!) allows Google to move toward a more iOS-App-Store-like level of "preventing users from hurting themselves" without much effort on their part, and without the backlash they'd receive if they did so as an end unto itself. (Contrast: the backlash when Microsoft tried that in Windows 8 with an app store containing only Metro apps.)

I expect that the user experience would be that, on Fuchsia-based devices, you'd have to either click into a "More..." link in the CWS-branded-as-Play-Store, or even turn on some setting, to get access to the "legacy" Play Store, once they deprecate it. It'd still be there—goodness knows people would still need certain abandonware things from it, and be mad if it was just gone entirely; and it'd always need to stick around to serve the devices stuck on "old Android"—but it'd be rather out-of-the-way, with the New apps (of which old Chrome Apps from the CWS would likely be considered just as "new" as newly-published Fuchsia apps upon the store's launch) made front and centre.

> Seems more likely they'll allow HTML apps into the Play Store, eventually getting rid of the Web Store entirely.

I would agree if this was Apple we were talking about (who is of a "native apps uber alles" bent) but this is Google. Google want everyone to be making web-apps rather than native apps, because Google can (with enough cleverness repurposed from Chrome's renderer) spider and analyze web-apps, in a way it can't spider and analyze native apps. Android native apps are to Google as those "home-screen HTML5 bookmark apps" are to Apple: something they wish they could take back, because it really doesn't fit their modern business model.

Lol, citation needed.

I was under the impression that MacOS was built from a POSIX kernel, all the way down.

XNU does embed BSD (and so POSIX) semantics into the kernel—some of which are their own efficient primitives, since there's no way to efficiently implement them in terms of Mach. But whatever BSD syscalls can be implemented kernel-side in terms of Mach primitives, are.

That would be lost which is a huge deal. The new gnu/Linux on ChromeOS would be fine as it runs on a VM and would still work.

Now they could move Android to using a VM but that it less efficient and most importantly takes more RAM and CBs do not normally have a ton of RAM.

In fact, starting with Android 7 they have started to lock down NDK apps that try to use APIs not listed as stable.

I am not convinced a micro kernel would be able to achieve.

Because POSIX has horrible security properties, and does not provide enough guarantees to create truly robust software. See for instance, the recent article on how you simply cannot implement atomic file operations in POSIX -- sqlite has to jump through 1,000 hoops to get something pretty robust, but it shouldn't be this way.

You don't need async for a proper concurrent system. Systems have been concurrent before async IO. The trick is that when a process does an IO you yield it and put it on a wait list and run literally anything else until the kernel receives the OK from the hardware control and resumes the process.

Using Async IO in Linux merely means your specific thread won't be immediately suspended until you get data (or it's in a reasonably close cache)

It would be quite silly if Linux would wait for every IO synchronously, any single core system would immediately grind to a halt.

  > Linux kernel has Async IO

In a fully async platform you would be able to do general-purpose programming without ever needing to use multithreading.

Example of a situation you can't do in linux: have a program doing select(2) or equivalent on both keyboard input and network input, in a single thread.

Since linux does not support this, you are steered to adapt solutions that are more complicated than a pure async model would be,

* Spin constantly looking for activity. These heats up your computer and uses battery.

* Have short timeouts on epoll, and then glance for keyboard input. This leads to jerky IO.

* Have child processes block on these operations and use unix domain sockets to feed back to a multiplexor (fiddly, kernel contention).

* The child-process thing but with shmem (fiddly)

* Something equivalent to the child process thing, but with multiple threads in a single process. (fiddly)

You would think that x-windows might help out here. What if you had a socket to X, and then multiplexed on that, instead of looking for keyboard input from a terminal? This opens new issues: what if X has only written half of an event to your socket when select notifies you? Will your X library handle this without crashing?

Rurban's comment above is correct. Linux is not async throughout.

On OSs that offer kevent you can get a fair bit further, but (I believe) you still can't do file creation/deletion asynchronously.

You're not the only task running.

Or do you expect Linux to pause the system when you read a file until the DMA is answered?

Linux itself is fully async, anything blocking (even interrupts to some extend) will be scheduled to resume when the reason it blocked is gone or alternatively check back regularly to resume.

A program running on Linux can do a lot of things async as mentioned via POSIX AIO. It's not impossible, go seems to do fine on that front too (goroutines are put to sleep when doing blocking syscalls unless you do RawSyscall).

The conclusion that lack of 100% asynchronity means it can't be properly concurrent is also wrong. As evident that sending something to disk also doesn't not halt the kernel.

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/aio....

But Async IO is part of POSIX.

I was mostly responding to the comment which cited synchronous IO as a problem with POSIX. Linux is most widely used so on top of being my primary domain on the issue, it's going to ensure best understanding.

See here: https://linux.die.net/man/7/aio

AFAIK, system-specific APIs such as epoll or kqueue are the way to go if you're serious about performing asynchronous IO?

The goal is to avoid blocking, not to offer async also. That's a lost cause.

Fuchsia once had the goal to offer async only, but then some manager decided that he needs some blocking, and then it was gone.

non-blocking only is lower level than blocking. You can always wait indefinitely for that callback, but usually you have a default timeout. L4 for example offers that API.

POSIX is a lowest common denominator API, its under specified and loosely interpreted. Which follows from its initial goals, which was basically to specify the bits common between various UNIX implementations. Those implementations obviously didn't want to change their implementation to match a rigid standard, so a lot of wiggle room exists in the standard.

The end result is that is pretty much useless for anything beyond "hello world" kinds of applications, both in terms of portability, as well as actual behavior (I could list a lot of cases, but lets leave that to google, with the starting idea to look at a couple posix API's, say close()'s errno's and the differing cases and what causes them on different OS's/filesystems). That is why there isn't a single OS out there that is _ONLY_ POSIX compliant. You need look no further than the 15 year old https://personal.opengroup.org/~ajosey/tr28-07-2003.txt and consider that the gap has widened as more performance or security oriented core API's have been introduced and linux's POSIX layer is further refined upon them.

Plus, the core API's in no way reflect the hard reality of modern hardware, leaving the standard even more under specified in the case of threads and async IO, which have been poorly bolted on.

Then there are all the bits everyone ignores, like the bits about the posix shell (ksh), and how certain utilities behave, while completely ignoring important things like determining metadata about the hardware one is running on.

I leave you with: https://stackoverflow.com/questions/2693948/how-do-i-retriev...

and https://stackoverflow.com/questions/150355/programmatically-...

Which is pretty basic information about a modern machine.

Only useful for CLI tty old style apps and daemons.

POSIX says that usernames and uids can overlap, and if a program takes either, it should accept the other a well. And if something could be a username or uid, it should be presumed to be a username, and resolved to a UID.

Now assume you have a user "1000" with uid 2000, and a user "2000" with uid 1000... you can see where this goes.

And this is why the BSDs have broken POSIX compatibility and require uids to be prefixed with # when they’re used on the CLI.

Nor is what you claim about the BSDs true. Aside from the fact that using the shell's comment character would be markedly inconvenient, especially since the colon is a far better choice for such a marker (c.f. Gerrit Pape's chpst), the OpenBSD and FreeBSD implementations of chown and newgrp do not have such a marker either documented or implemented, and in fact operate as the SUS says.

User names and user IDs are, by contrast, explicitly called out by the standard as strings and integers, two quite different things which have no overlap.

* http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...

* http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...

What a nice analogy.

I quite like Linux but I would not say the current state of the core kernel is "really" nice. Somewhat nice, maybe, but "really" nice? You just have to look around to see exemples of non-nice things (too much things in some headers, jumping between source files back and forth for no reason, excessive use of hand coded vtables, internal "framework" that are not as simple as they should, at other time lack of abstractions and excessive access to internals of structures, etc). Granted, it is way less buggy than some other softwares, but I think I'll know when I read a really nice core, and for now I've never had the feeling that Linux has one.

Still find it quite good, to be clear.

https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02...

I agree that there is much to be explored yet in OS design. However it needs deep pockets to actually support wide array of hardware. Or a long time.

I cheer for Toybox and Oil shell regarding the GNU userland.

When I see all the openat(2) and friends family of functions or chmod/fchmod, chdir/fchdir, barbaz/fbarbaz I'm thinking that it all needs a bit of a clean up. Some batching system call would be appreciated as after Meltdown syscalls are more expensive.

I personally like and keep my fingers crossed for DragonflyBSD. They are doing things that are innovative while keeping conservative core for the lack of better words. But at the same time DragonflyBSD has minuscule hardware support comparing to Linux.

I'm not convinced that even Google has the resources to build something like that successfully. Except perhaps if they target a very specific use case.

Google CAN'T do something small, simple and elegant, partly for the very reason that they're too big.

So, quite a lot if that experience is anything to go by.

They obliged with a full crate of first generation Motorola phones, each one preloaded with a 30 day free Verizon plan. Every person who took that class got one, and surely all of them made it back into the school's hands at the end of the quarter.

(I'm not sure how many people actually ever compiled and executed any go binaries on arm that year; we all learned Go, and it was a great class! But as far as the class, the phones were completely unnecessary. I think that they did make a more relevant class where the phones were able to be used again the year after that.)

Had Go been released at AT&T and it would have shared the same fate as Limbo.

This is beyond belief. What companies are using Go with the hopes of being acquired by Google? Does anyone honestly believe that Google's acquisitions teams know or care about programming languages? Any business that acquires companies on that basis is doomed to failure, as is any company that hopes to be acquired on that basis.

Systems does mean that. There are 2 broad categories of software you can write. One is software that provides a service to the user directly. That is an application. The other kind is software that provides a service to applications. That's systems software. Do you think there's something wrong with this notion? It's pretty well accepted over the decades:

http://en.wikipedia.org/wiki/System_programming

https://fuchsia-review.googlesource.com/q/status:open

I'm certain you can find complexities and bloat in there too, if you look.

I don't have much experience with them but I think VxWorks and QNX are good examples of simpler OSs. I do have some experience with Minix3 and it is certainly one. I guess the BSDs stand somewhere in the middle.

OS X has tons of weird quirks and compatibility mindfucks going back to their transition from OS 9 (all those .DS_Store and _filename files for "resource forks")

Backwards compatibility is #1 reason for increasing complexity. Apple is sometimes good in cutting away compatibility for the sake of cleaning up, but there are still weird issues poking now and then

hell the whole NSEverything is a compatibility thing with NextStep, which is long dead.

Which is the worst of both worlds.

It might not be backwards compatible but from a developers perspective it is nice to be able to reuse old knowledge.

I think Apple has been much better than MS or Linux in continously modernizing and upgrading what they have. On windows and linux things tend to become dead ends as new flashy APIs appear.

Sure old win32 and motif apps might still run but nobody really develops using these APIs anymore.

On windows I first use win32, then MFC, then it was WinForms. Then all of that got depricated and we got WPF, silverlight and then I sort of lost track of what was going on. Meanwhile on Linux people used tcl/tk for GUIs early on. And there was motif, wxeindows. KDE and Gnome went through several full rewrites.

If we look at MacOS X as the modern version of NeXTSTEP the core technology has been remarkable stable.

Sure they have broken compatibility plenty of times but the principles and API are at their core the same.

One just needs to look into the right spot.

Didn't Mach start out as a replacement kernel for BSD? Building a Unix on top of Mach is like building a truck over truck chassis.

Next's VP of Software Engineering, Avie Tevanian, was one of the Mach project leads. Richard Rashid, who lead the Mach project ended up running Microsoft Research's worldwide operations.

Their work on a virtual memory subsystem got rolled back into BSD.

https://en.wikipedia.org/wiki/Mach_(kernel)

The Computer History Museum has an interesting long form interview with Avie:

Part 1: https://www.youtube.com/watch?v=vwCdKU9uYnE Part 2: https://www.youtube.com/watch?v=NtpIFrOGTHk

In late 80's/early 90's there were various projects that attemted to build unix on top of that as true micro kernel architecture with separate servers for each system service. Performance of such design was horrible and there are two things that resulted from that that are still somewhat relevant: running whole BSD/SysV kernel as Mach task, which today means OS X and Tru64 (at the time both systems had same origin as both are implementations of OSF Unix) and just ignoring the problem which is approach taken by GNU/Hurd.

Worth noting that's because Mach IPC is extremely slow, an order of magnitude slower than L4 family.

https://www.youtube.com/watch?v=B65ND8vUaKc

In theory it makes me wonder if iphone's storage is slightly faster than the latest galaxy or if the process by which one loads an iphone app is slightly faster/more efficient than the one by which an android app is loaded or if the tiny selection of apps the reviewer picked are just better optimized for iphone. Nobody smoked anyone and nothing of note was learned by anyone. So much so that I wonder why you bothered to watch said link or paste it here.

I said half an OS because its built on technologies like linux and java not because its half assed even though it is.

Certain apps were slower to load on the Samsung device. Additionally, the Samsung device encoded the 4K siginificantly video faster and took round 1 by 14 seconds and round 2 by 16 seconds.

>In theory it makes me wonder if iphone's storage is slightly faster than the latest galaxy or if the process by which one loads an iphone app is slightly faster/more efficient than the one by which an android app is loaded or if the tiny selection of apps the reviewer picked are just better optimized for iphone. Nobody smoked anyone and nothing of note was learned by anyone. So much so that I wonder why you bothered to watch said link or paste it here.

The iPhone X has faster storage and a significantly faster SoC. A 30 second win by the Samsung phone is what I could call getting smoked.

>I said half an OS because its built on technologies like linux and java not because its half assed even though it is.

And iOS is was a decedent of MacOS which itself is a decedent of NeXTSTEP. It also uses a language 11 years older than Java. So it sounds like iOS also meets your criteria of being "half assed".

When I saw your comment I immediately was reminded of Joel on Software: "The single worst strategic mistake that any software company can make is to rewrite the code from scratch."[1]

[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-...

1) Get yourself placed on the "Legacy team" that is supposed to "minimally support the application while we transition to the new system".

2) Do whatever the hell you want with the code base (i.e., refactor as much as you want) because nobody cares about the boring legacy system.

3) Respond directly to user/customer needs without having to worry about what upper management wants (because they are distracted by the beautiful green-field rewrite).

4) Retain your job (and probably get a promotion) when they cancel the rewrite.

1) Get stuck on the "Legacy team", as expensive contractors are called in to produce the rewritten version from scratch in an unrealistically small fraction of the time the original took.

2) Be told you can only fix bugs (with more layers of short term hacks), not waste time with refactors that have "no business value" for a code base that will be scrapped "soon".

3) Don't add any new features, to prevent the legacy system becoming a perpetually moving target that would delay the beautiful green-field rewrite even longer.

4) Hate your job, and watch your coworkers leave in disgust, further starving the team of resources, until you end up quitting too.

The programmers with the political skills to get put in the "Rewrite team" will have had their jobs in the "Legacy team" protected in case of project cancellation. Or they will knife you in the back to get their old jobs back -- they will know about the cancellation and have plenty of time to maneuver before you know what's going on.

1) Get on the new team

2) Ensure to get into the more interesting tech modules

3) Improve your CV

4) Use your soft skills to mingle with everyone and be aware of where the wind blows

5) In case of iceberg ahead, jump ship to a better job taking advantage of the new skills

1) Do a startup

1) Quit job 2) move to the woods 3) shun all electrical devices 4) mediate until you become one with the universe

Firefox started as a fresh UI reskin for the old browser interface, and indeed continued as "mostly a skin" for years (incidentally, so did Chrome).

(You can still get the non-firefox skin incidentally, it's "Mozilla Seamonkey")

Then the Rust rewrite. Rust was a hobby project, and they rewrote layout engine interface, and nothing more than that. Then CSS.

Now it's an HTML parser, a layout engine, a parallel CSS engine, and a GPU webpage renderer (still "missing"/non-rust are 2d compositing and javascript). Each of those components replaced another in a working version and there were at least beta versions of firefox of all the parts.

If you're Google and you have people with these abilities kicking about, it's probably not a crazy investment to see what happens. We've got a HN story elsewhere in the list on post-quantum key agreement experiments in Chrome, again there's a fair chance this ends up going nowhere but if I was Google if throw a few resources at this just in case.

But on the whole I expect Fuchsia to quietly get deprecated while Linux lives on, even if there's lots to like about Fuchsia.

Link for reference: https://news.ycombinator.com/item?id=16811554

A rewrite made no sense to me since I'd end up maintaining version A alongside version B with B constantly lagging A unless I severely restricted the scope of B in which case it'd be an incomplete (though better written/more maintainable A).

Instead I went the isolate (not always easy), shim, rewrite, replace, remove shim approach.

It does feel a bit like spinning plates blindfold sometimes in the sense I'm always expect to hear a crash.

So far I've replaced the auth system, the reports generation system, refactored a chunk of the database, implemented an audit system, changed the language version, brought in proper dependency management, replaced a good chunk of the front end (jquery soup to Vue/Typescript), rewritten the software that controls two production units and implemented an API for that software so that it isn't calling directly into the production database.. and done it without any unplanned down time (though I'm still not sure how - mostly through extensive testing and spending a lot of time on planning each stage).

It's slower because I have to balance new features against refactor time but I have management buy-in and have kept it, mostly through been extremely clear about what I'm working on and what the benefits are and have built up some nice momentum in terms of deploying new stuff that fixes problems for users.

The really funny part is that even though I'm re-factoring ~40% of the time I'm deploying new features faster than previous dev who wrote the mess...because I spent the time fixing the foundations in places I knew I'd need for new features going forwards.

Althought in OS space one might argue that second generation of time sharing OSes (TOPS-10, ITS, MCP...) got more things right than are right in Unix and such.

It's not just the kernel design, it's the driver design as well. Fuschia has drivers being ELF binaries that hook into Device Manager process[0]. I believe they are developing a standard API for how drivers interact with Fuschia.

Linux doesn't really have this today, which means that the driver must be in the kernel tree to have it keep up with the kernel's interface. And with ARM lacking something like a BIOS for phones, each chip that Qualcomm et al make requires a full BSP to get it working. And from what I understand, Linus and others don't want these (relatively) short-lived processors to be checked into mainline linux (plus you have the lag time of that code being available). Android's Project Treble[1] aims to address this some by creating a stable API for hardware makers to use to talk with Linux, but it is not an ideal solution.

[0] https://fuchsia.googlesource.com/zircon/+/HEAD/docs/ddk/over...

[1] https://android-developers.googleblog.com/2017/05/here-comes...

[0] https://en.wikipedia.org/wiki/Device_tree

I don't think that's true. e.g. 4.17 is accepting code to run Samsung Galaxy S3 (released nearly 5 years ago). It is left to volunteers since most vendors dump a source release to comply with the GPL but themselves don't upstream it to Linus.

But, the core idea behind SBSA and SBBR is to form a common platform to which the vendors conform their machines so they don't have to keep up-streaming special little drivers for their special little SOCs. Only time will tell if its a success, but a large part of the community has effectively declared that they aren't really going to conform to the ideals of a standard platform. So, the ball keeps rolling and new DT properties keep getting added, and new pieces of core platform IP keep showing up. At this point arm64, despite just being a few years old already looks as crufty as much older architectures with regard to GIC versions, dozens of firmware interfaces, etc due to the lack of a coherent platform/firmware isolation strategy.

Ugh, tell me about it. Just dealing with a relatively simple situation where you've got signals coming in while reading a pipe is a hassle to get completely correct.

I am so there for an operating system API that is relatively simple and sane. Where writing correct programs is, if not the easy path, at least not the hard and obscure path.

Of course, it's pretty hard to deal with a setup like that from plain C + libc + libpthread code, so OSes shy away from it. But if your OS also has OS-global sandboxed task-thread pools (like macOS's libdispatch)—and people are willing to use languages slightly higher-level than C where tasks built on those thread-pools are exposed as primitives—then it's not out of the question to write rather low-level code (i.e. code without any intermediating virtual-machine abstraction) that interacts with such a system.

Wasn't QNX like that?

Edit: in essence QNX messages work like syscalls, with the difference that there are multiple "systems" that accept "syscalls". For what it's worth Solaris has quite similar IPC mechanism that is mostly unused.

Sure, you can use ALPC, but only by using undocumented unstable NT API...

Sigh.

[1]: http://cr.yp.to/docs/selfpipe.html [2]: http://man7.org/linux/man-pages/man2/signalfd.2.html

That can't be very good for debuggability.

Mainly, I'm just getting tired of people trying to cut things out of the solution by cutting them out of the problem.

Saying that it's possible to solve a problem in a simpler fashion is fine. Saying a problem shouldn't be solved, that nobody should have that problem, so therefore we won't solve it, is not fine if you then turn around and compare your cut-down solution to the full solution.

It's possible to be more efficient at showing a bit of text and graphics which is what mobile phones do a lot more of than raw number crunching, except for games, except for games of course.

Not sure about display though, but don't expect monolithic kernel to help here somehow either.

GPUs don't have much security at all, there is lots of DMA or mapped memory. Though on most modern monolithic kernels a lot of this work is either in modules (AMDGPU on Linux is usually a module not compiled into the kernel) or even userspace (AMDGPU-Pro in this case). Mesa probably also counts.

Microkernels aren't the ideal kernel design. Monolithic isn't either. I put most of my bets on either Modular Kernels if CPUs can get more granual security (MILL CPU looks promising) or Hybrid Kernels like NT where some stuff runs in Ring 0 where it's beneficial and the res in userspace.

Of course they can share an interface, I even pointed out the vale switch as an example of this [1]. And it is very fast.

The thing is isolation and granularity that microkernels happen to have force certain design and implementation choices that benefit both performance and security on modern systems. And monolithic kernels while theoretically can be as fast and as secure actually discourage good designs.

[1] http://www.openvswitch.org/support/ovscon2014/18/1630-ovs-ri...

I also severely doubt that microkernels encourage efficient design. I'll give you secure but it's not inherent to microkernels either (NT is a microkernel, somewhat, and has had lots of vuln's over the years, the difference between microkernels and monolithic or hybrids like NT is that most microkernels don't have enough exposure to even get a sensible comparison going)

IMO microkernels encourage inefficient designs as everything becomes IPC and all device drivers need to switch ring when they need to do something sensitive (like writing to an IO Port unless the kernel punches holes into ring 0 that definitely don't encourage security).

Monolithic kernels don't necessarily encourage security but definitely efficiency/performance. A kernel like Linux doesn't have to switch priv ring to do DMA to the harddisk and it can perform tasks entirely in one privilege level (esp. with Meltdown, switching ring is a performance sensitive operation unless you punch holes into security).

I don't think monolithic kernels encourage bad design. I think they are what people intuitively do when they write a kernel. Most of them then converge into hybrid or modular designs which offer the advantages of microkernels without the drawbacks.

A monolithic, hybrid or modular kernel can significantly reduce this overhead while still being able to employ the same methods to amortize the cost that exists.

A microkernel is by nature incapable of being more efficient than a monolithic kernel. That is true as long as switching processes or going into priv has a non-zero cost.

The easy escape hatch is to allow a microkernel to run processes in priv ring and in the kernel address so the kernel doesn't have to switch out any page tables or switch privs any more than necessary while retaining the ability to somewhat control and isolate the module (with some PT trickery you can prevent the module from corrupting memory due to bugs or malware)

Consider showing a webpage. You have a network stack, a graphics driver, and the threads of the actual browser process itself. It's substantially easier to about bottlenecking through one or more locks (for, say an open file table, or path lookup, etc) when the parts of the pipeline are more separated than a monolithic kernel.

Lock-free concurrency is also achievable.

Again, this isn’t the user land we’re talking about, in the sense that the kernel is expected to be highly optimized.

Granted, a multi process architecture does have virtues, like stability and security. But performance is not one of them.

Lock free concurrency is typically via spinning and retrying, suboptimal when you have real contention. It's better not to contend.

Kernel code isn't magic, its performance is dominated by cache just like user space.

High performance applications get the kernel out of the way because it slows things down.

Lock free concurrency is typically done by distributing the contention between multiple memory locations / actors, being wait free for the happy path at least. The simple compare-and-set schemes have limited utility.

Also actual lock implementations at the very least start by spinning and retrying, falling back to a scheme where the threads get put to sleep after a number of failed retries. More advanced schemes that do "optimistic locking" are available, for the cases in which you have no contention, but those have decreased performance in contention scenarios.

> Handing off means to stop using it and letting someone else use it. Only copy in rare cases.

You can't just let "someone else use it", because blocks of memory are usually managed by a single process. Transferring control of a block of memory to another process is a recipe for disaster.

Of course there are copy on write schemes, but note that they are managed by the kernel and they don't work in the presence of garbage collectors or more complicated memory pools, in essence the problem being that if you're not in charge of a memory location for its entire lifetime, then you can't optimize the access to it.

In other words, if you want to share data between processes, you have to stream it. And if those processes have to cooperate, then data has to be streamed via pipes.

> High performance applications get the kernel out of the way because it slows things down.

Not because the kernel itself is slow, but because system calls are. System calls are expensive because they lead to context switches, thrashing caches and introducing latency due to blocking on I/O. So the performance of the kernel has nothing to do with it.

You know what else introduces unnecessary context switches? Having multiple processes running in parallel, because in the context of a single process making use of multiple threads you can introduce scheduling schemes (aka cooperative multi-threading) that are optimal for your process.

It's not an automatic security win.

So from a security standpoint MacOS X is mostly in the kenel camp, not the microkernel one.

The project at Carnegie Mellon ran from 1985 to 1994, ending with Mach 3.0, which is a true microkernel. Mach was developed as a replacement for the kernel in the BSD version of Unix, so no new operating system would have to be designed around it. Experimental research on Mach appears to have ended, although Mach and its derivatives exist within a number of commercial operating systems. These include all using the XNU operating system kernel which incorporates an earlier, non-microkernel, Mach as a major component. The Mach virtual memory management system was also adopted in 4.4BSD by the BSD developers at CSRG,[2] and appears in modern BSD-derived Unix systems, such as FreeBSD.

Say what you will about how terrible POSIX is, Be recognized the tremendous value in supporting POSIX: being able to support the mounds and mounds of software written for POSIX. It chose to keep enough POSIX compatibility to make it possible to port many common UNIX shells and utilities over, while Be developers could focus on more interesting things.

So where were the problems?

One huge problem was drivers, particularly once BeOS broke into the Intel PC space. Its driver interface and model was pretty darn slick, but it was different, and vendors wouldn't support it (take the problems of a Linux developer getting a spec or reference implementation from a vendor and multiply). This cost Be, and its developer and user community, quite a bit of blood, sweat, and tears.

Another big problem was networking. Initially, socket FDs were not the same thing as filesystem FDs, which had a huge impact on the difficulty of porting networked software over to BeOS. Eventually, BeOS fixed this problem, as the lack of compatibility was causing major headaches.

The lesson: if you are looking to make an OS that will grow quickly, where you will not be forced to reinvent the wheel over and over and over again, compatibility is no small consideration.

Yes the Linux kernel is bloated, bloated with tons of code responsible for making it work on exotic hardware. Yes x86 is bloated, let's just remove those useless AVX instructions. Yes MS Excel is bloated, who the hell is using all those features [1]?!

You can only have two alternatives : either your software is “bloated”, or then it will be replaced by something else, which is more “bloated” and works for more people.

Notice than I'm just criticizing the “bloat” argument, I'm just not criticizing Google for creating a new OS from scratch, which can bring a lot on the table, if done properly and includes innovations from the past 30 years, like was done when creating Rust for instance.

[1]: https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv...

https://doc.lagout.org/ looks quite shady, so I guess someone uploaded their copy without copyright holder's permission.

A truck is not more complex than a car. A truck is bigger because it is designed to carry more load.

https://www.linux.com/news/event/open-source-summit-na/2017/...

In the video, it is show that the binary size of "core" linux has barely grown

https://youtu.be/ynNLlzOElOU?t=14m45s

I don't know what this thing is, but it doesn't look very appealing.

https://en.wikipedia.org/wiki/Fiddlehead_fern

1. They want a modular microkernel OS with modern capabilities.

2. They want to eventually replace their internal Linux based OS with Fuchsia.

3. Fuchsia, or parts of it, will be the basis for all of their products.

4. They want to control the direction of their own OS.

Problem solved!

One of the advantages of a microkernel architecture is that it is relatively easy for different components to be written in different languages.

And the borrow checker, along with move semantics by default and the Send and Sync traits, help with several other aspects of safety as well; the Send and Sync traits encode information on what can safely be moved or shared between threads, and move semantics by default (and checked by the compiler, instead of at runtime as in C++), make it easier to encode state transitions on objects in ways that you can't try to perform operations on an invalid state.

But as others point out, Zircon, the Fuchsia kernel, was written before Rust 1.0 was released, and even after Rust 1.0 was released and stable it was still a bit rough to work with for a little while.

If you were starting a new project from scratch today, I'd seriously ask why not Rust, though of course there are other reasons why you might not choose it. But given the history, and how new Rust was at the time when this project was started, it makes a lot of sense.

While I'm not saying that rust is bad, Rust as a kernel language won't help you here. The simple proof is that Redox-os (which is a microkernel built in Rust) was found with the same class of bugs your average OS had.

Source? I haven't heard about these vulnerabilities.

In a kernel, there are several places where Rust won't help you. There are certain things you have to do that are going to need unsafe code, and need to be audited just as much as code written in any other language. The kernel is also a place ripe for logical vulnerabilities to creep in; it is the main arbiter of what is allowed and is not allowed from processes, so many possible vulnerabilities are logical, not memory safety issues.

On the other hand, when a kernel gets complex enough, there are places where memory safety or thread safety issues can come up even in code which doesn't require particular unsafe features and which is not involved in a security role; just things like manipulating complex data structures in multiple threads. This kind of code is the kind of code in which Rust can help catch issues; and it definitely still happens even when very good programmers, with good code review, work on the kernel.

Rust is not a panacea; it is not a cure-all. But it reduces the attack surface considerably, by containing the amount of code that is exposed to certain very common classes of bugs.

Edit: to clarify ... I don't mean that they won't be able to understand the concepts ... I mean that they will lose a lot of productivity up front and won't appreciate the backend gains enough for there to be a political consensus to switch.

https://dlang.org/spec/betterc.html

Fuschia will become another monolith where the assumptions no longer matter given tech of 2035 or whatever, and something new will happen

IMO the better model would be a simple hardware BIOS type thing with a standardized build system embedded that enables completely custom environment builds from whatever source repo people want

These big monoliths seemed more valuable in a pre-internet is everywhere world, so they include the kitchen sink too

I’m talking to the commentary about a dated cultural artifact sucking, championing of new hotness, a new culture that will develop around it and eventually fade as technology changes and the peddlers of the latest and greatest will largely say the same “wtf @ this mess”

But downvote reality of human progress that has occurred over and over

The opinionated nature of humans is bizarre, given that we keep, in a social way, repeating our dead relatives

We can fix the future from a past that won’t live to see it

It’s all nonsense for the majority of humans, but this little microcosm called HN has its finger in the pulse of what we’ll need in 2040!