I also submit that to understand why that happened, one needs to consider the context at the time that the decision was made:
http://www.h-online.com/open/features/GNU-HURD-Altered-visio...
http://www.groklaw.net/article.php?story=20050727225542530
« RMS was a very strong believer -- wrongly, I think -- in a very greedy-algorithm approach to code reuse issues. My first choice was to take the BSD 4.4-Lite release and make a kernel. I knew the code, I knew how to do it. It is now perfectly obvious to me that this would have succeeded splendidly and the world would be a very different place today.
RMS wanted to work together with people from Berkeley on such an effort. Some of them were interested, but some seem to have been deliberately dragging their feet: and the reason now seems to be that they had the goal of spinning off BSDI. A GNU based on 4.4-Lite would undercut BSDI.
So RMS said to himself, "Mach is a working kernel, 4.4-Lite is only partial, we will go with Mach." It was a decision which I strongly opposed. But ultimately it was not my decision to make, and I made the best go I could at working with Mach and doing something new from that standpoint.
This was all way before Linux; we're talking 1991 or so. » -- Friar Thomas Bushnell
They looked at BSD. The BSD people were not sure, so RMS decided on another path.
If GNU had picked the BSD kernel, there could have been a working Free xNix before Linux and things would have been very different.
Secondly, I think it's important when discussing microkernels and microkernel OSes to consider more than the most famous one: Mac OS X.
Xnu is based on Mach, but it's not a pure microkernel: its Xnu kernel contains a large, in-kernel "Unix server" derived from BSD code. This was done for performance reasons – remember, macOS is Mac OS X is NeXTstep, written for a 25MHz 68040 in around 1987-1988.
There are better examples. QNX is probably the best: a working, true-microkernel, Unix-like OS, dating from 1982. At one point shared-source but not any longer.
There is also Minix 3, which is a different OS from Minix 1 & 2, the OS that inspired Linux and upon which Linux was initially bootstrapped.
Minix 3 is still quite limited: no SMP, some missing APIs etc. But QNX proves that a true microkernel, on generic hardware such as x86 and ARM, can support SMP and so on.
> I had hopes in the back of my head that "maybe it'll get better/more ergonomic in the next few years".
I feel much the same way about wanting to run something like FreeBSD and having it just work, as opposed to running into weirdness because of the driver situation, with which GNU/Linux seems to be getting better at (even though you sometimes are forced to install proprietary ones for optimal experience, should get better in the next decade).
So, might have to wait for a bunch more years, or just pick something else, like OpenBSD, or just run in a set of constraints for having a really predictable and well supported hardware configuration, which isn't always possible. Alas, my off brand Polish netbook will just have to wait before i can (hopefully) run BSD on it comfortably. Well, short of making the missing pieces of puzzle myself, for which i'd probably also need a decade or so of systems level development experience, as opposed to just web dev and occasionally dabbling in lower level stuff to mediocre success. Of course, by then the netbook itself might not be relevant, so who knows.
I also feel much the same way about GNU/Hurd - a project that is conceptually interesting, but isn't yet quite stable, even though apparently you can get Debian for it: https://www.gnu.org/software/hurd/
Now, i don't have almost any experience with it, apart from reading about it, but some people tried figuring out when it could be released based on the bug reports and how quickly they were being addressed, and the number that they came up with was around 2070 or so.
In summary, there are probably projects out there, for which their age isn't necessarily related to how usable they are, whereas other pieces of software will have never truly achieved that stability in the first place. Not all old software is good (edit: at least not for certain use cases).
Of course, there are exceptions, for example, you can look at Apache2/httpd: it is regarded as old and dated for the most part, however just recently an update was released, which added mod_md. It now lets you get Let's Encrypt certificates without external tools (like Certbot), in some ways setting it ahead of Nginx: https://httpd.apache.org/docs/2.4/mod/mod_md.html Not all old software is always boring.
Same for Docker and any other tools. If developers use tool A for use case X, instead of tool B, then maybe there are some very good reasons for this widespread usage? That's also probably the answer to this debate - regardless of their conceptual/technical benefits, the usability will probably decide which tool will win in the long term.