... but I think the systems built "by hackers for hackers", nix clones in general and linux distributions in particular, are generally the best stuff out there.

It's all the attempts to be "intuitive" and "easy" and "just work" that really fails. It's the stuff "by designers for users" that sucks.

The old-ish core unix/linux stuff has been slowly polished over the years and works pretty damn well, and you depend on its quiet stable operation for all your fancy user-friendly (and "dev-friendly") layers that need to be completely replaced every couple of years, for your favorite phone and your favorite websites.

EDIT: you complain about trying to compile kdevelop, on ubuntu... well there's your mistake, both of those things are trying hard to be user-friendly. Try plain debian, plain vim, etc.

This is oversimplified, but I think most of the things today that try to be user-friendly and intuitive are stuck in the second phase--it takes a lot of effort and care to get to the third one. The iPhone interface (touch, swiping, ...) might actually be an example of a successful attempt at the "just works"-category--we've all heard of babies learning to navigate iPads etc. And today it seems so obvious, you don't even think about all the nuances that have to be right for this to work, but that's probably a hallmark of the designs that really are intuitive--they seem so obvious you don't immediately realize someone had to come up with them, and iterate on them until they just worked.

Hacker personalities tend to fall into conflict with that ambition(even if they enjoy the results) since "hackability factor" is dependent on having something to configure, and you can't configure a nothing that just works - especially since in the space between "configure everything" and "just works", you have a form of the uncanny valley effect where the experience gets a lot worse and there is no possibility to configure yourself out of it. That stops UX from enjoying simple incremental improvements.

The very UNIX-y solution is to have a nice, simple, friendly UI on top of the infinitely-configurable command-line engine. Or better yet, two different friendly UIs and you choose which one you like better.

You. Don't. Want. Infinitely. Configurable.

What you do want is lots of simple, friendly tools with limited configurability that you can connect to solve larger issues.

To compare it to language, you don't want to have two great novels that are paths through "choose your own adventure". You want words that work together, that are composable according to well-understood, simple rules.

Whenever a system like that comes along (unix pipes, REST APIs), people can build amazing things on top of that. The infinitely configurable hypercomplex thing idea gives you XSL:FO, XSLT, the W3C, C++, and assorted fun.

Steve Jobs solution was to make a simple 80% solution. And of course that works in the consumer market for a great many things, particularly innovative new products. However some things are irreducibly complex and still require a solution. In these cases visionaries fail us, and what we need it is practical iteration to get us a working solution. The end result is always ugly, but it works. Until we have bigger brains I don't see how we can avoid this fundamental limitation in our ability to build systems.

At least, that's what I like to think.

Relatively speaking, humans have been thinking about interaction with computers for a very short time, so maybe there's hope in the future :).

"Unix has retarded OS research by 10 years and linux has retarded it by 20." -- Boyd Roberts

It was the large body of useful open source and copyleft code. This made it so everyone could stop pouring huge resources into the base OS layer individually, use something that works really quite well (with whatever necessary tweaks since it is open source), and focus on what goes on top. And there's no one extracting rent on this layer. There's no licenses and activations. Nothing of the sort. (You can contract for RHEL, but you can just as easily not.)

BSDs came a couple of years after linux, and plan9 was open sourced many years after (and initially under the problematic "lucent public license"). It was way too late. And the innovations to the most basic interfaces of the OS were just not nearly as valuable as the body of open source which was already available for Linux / BSD.

Plan9 is like the Concorde - pinnacle of tech/design, nobody uses it. Never had seat-back entertainment. Never had wifi. Maybe a silly metaphor, but it fits the OP.

There are still low-level changes these days in Linux and BSDs. Nothing that drastically breaks compatibility of course. But more security boundaries and privilege management (openbsd w/x and aslr stuff, linux seccomp-bpf, freebsd capsicum, containerization), and speed/efficiency measures like sendfile(), epoll() / kqueue(), RCU fs cache lookups, etc.

Never had wifi.

Well, it was developed during a different time. It did get wi-fi thanks to 9front, though.

linux seccomp-bpf

A sandboxing/syscall filtering mechanism. Nothing new there, and its interface is rather leaky, much like Berkeley sockets are.

freebsd capsicum

This is one of the few legitimately interesting projects going on. Bringing capability-based security on top of fds. I don't know if it'll catch on beyond FreeBSD, though. Google seems to be an early adopter.

containerization

Nothing that IBM didn't do much better. Docker reeks of opportunism.

sendfile(), epoll() / kqueue()

sendfile(2) is just a cheap trick that got turned into a syscall: https://groups.google.com/forum/#!msg/golang-nuts/gdp1q6T0DN...

epoll(2) isn't anything special, it's a rather convoluted if performant I/O multiplexing mechanism. kqueue(2) is much cleaner designed, at least.

It's a shame, because Plan 9 had so many brilliant ideas, but once you have a mature ecosystem, incompatibility is a killer.

I wonder how well v9fs works...

-- Rob Pike on his Slashdot interview

The thing is that despite them bringing enormous economic benefits when realized in the mainstream, the short-term incentives are aligned to hacking around what's broken instead.

[1] https://www.usenix.org/legacy/publications/library/proceedin...

Many of these would be big wins for developers. But developers actually have an incentive to make their own job harder, as long as they charge for their skills. Tougher development serves as a barrier to entry for the profession, which increases the wages they can charge. Make an OS so developer-friendly that the end-user can develop and the profession of software engineer would go away...which may not actually be a bad idea in theory, but neither software engineers nor end-users seem to be that keen on it.

Well, not if you can also just restart to known-good states. Having a reincarnation server of some sorts is a given.

Not to mention "turn it off and on again" usually never works for Unix to begin with.

Dynamic code upgrading loses much of its benefit when users want to be prompted every time an upgrade is ready.

How does DSU hinder post-completion prompting?

But developers actually have an incentive to make their own job harder, as long as they charge for their skills. Tougher development serves as a barrier to entry for the profession, which increases the wages they can charge. Make an OS so developer-friendly that the end-user can develop and the profession of software engineer would go away...which may not actually be a bad idea in theory, but neither software engineers nor end-users seem to be that keen on it.

This is actually an interesting hypothesis. That a lot of inefficiency in software is really a form of job security.

It might make sense, but then again it seems like programmer wages dropping is inevitable, since software does not have scarcity unless you maintain it artificially. I'd wager FOSS has had an impact, though it obviously hasn't finished the job.

It's certainly possible this is why programmable UIs like Oberon and Cedar never caught on, though.

It doesn't hinder it, but post-completion prompting destroys a lot of the benefit.

Think about the user's perspective: they have already context-switched out of what they were doing to read the upgrade notice. Most apps these days either have auto-save or they're passive information-consumption apps, and so they're not going to lose work if the system shuts them down and restarts them. The OS can prompt them and restart the affected programs automatically after swapping out the binaries in the background, like how MacOS/Ubuntu/Google software updaters work.

So making it fully dynamic saves the user about 30 seconds every month, at the cost of a lot of complexity. There are much easier ways to save 30 seconds per month in user time.

This is actually an interesting hypothesis. That a lot of inefficiency in software is really a form of job security.

It doesn't even have to be deliberate. The only requirement necessary for this dynamic to emerge is to believe in the division of labor and have a mechanism for payment.

A certain segment of the developer population has a burning curiosity about how things work, all the way down. This segment is disproportionately represented among OS developers, because why else do you get into OS development if not to know how things work all the way down? The general public lacks this desire; most of them are quite happy to fork over money (or their personal data) to get the computer to do something useful to them. And a good portion of the developer population lacks it as well; many of them are quite happy to make what people want in exchange for money. The general public doesn't care about orthogonal persistence as long as they don't lose work when an app crashes. A commercial developer would almost rather it didn't exist, because then he can build auto-save functionality into his product and use it to differentiate from his competitors.

OSes that understand this dynamic and embrace it - like Microsoft, Apple, Android - tend to do much better than OSes that don't, like Genera, SmallTalk, or Oberon.

I sometimes wonder if devops is the developers way to get admins out of their way so they can work on the next shiny without the admin going "sorry, not until you can prove it is as stable etc as the one we already have".

-- sed, awk, and their friends.

(Hence to me there's no difference whatsoever between one Linux distro and another, because in any case I can only get plain vim and plain grep to work and not much else. So I won't reply wrt Debian vs Ubuntu except that the other standard reply that I get apart from "running old software" being wrong is, I always seem to be using the wrong distribution. I think the one thing that unites all the distros is that each of them is always the wrong one...)

The difference is that on Windows I can run VS just as easily as I can run vim.

Really, I'm just quite biased against windows in particular, because I had only Windows 98 and Windows XP when I first came of the age to have and tinker with my own computer, and the amount of stuff that you can't control and can't fix (installing common software, networking, file sharing, drivers) is infuriating. If something only runs on Windows, that's it's biggest flaw.

EDIT it's an easy potshot so I'll throw in this link http://visualstudio.uservoice.com/forums/121579-visual-studi...

But honestly all of that misses the point. The article is talking about typical users, not developers. Building stuff is far beyond people who don't know what a browser cookie is.

I spent hours today trying to set up prosody XMPP on a local LAN and found I couldn't do it with the server software that promises to have you 'up and running in minutes'. It made me think that perhaps in the same way you have test cases for features, you might have test situations to determine what features you should have in the first place. Does your software work if I'm in environment X? How about environment/situation Y?

I at this point feel so disgusted with the entire experience that I would prefer to just write my own, the cost of creating a new code has become lower than fully understanding an old one:

"It is easier to write a new code than to understand an old one." - John Von Neumann to Marston Morse, 1952 as quoted in Turing's Cathedral

EDIT (Mon Jul 20 23:30:32 PDT 2015): Lots of people citing Nick Bostrom on the AI stuff, for a different perspective I'd like to recommend Scott Alexander's (Looong) Meditations on Moloch: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/

"They broke their back lifting Moloch to heaven!"

Interestingly, I've found it to be the opposite. Most of systems built "by hackers for hackers" both have a surprisingly long (yet not obsolete) life, and are vastly more compatible across different environments. Emacs/vim has been around for a really long time, and there is no reason to believe they will disappear at this point. Similarly, I can comfortably switch from Linux to OS X to FreeBSD thanks to bash and the gazillion posix tools (grep, ps, awk...). The same definitely can not be said for Windows 7 to 8 so far.

As to easily switching... erm... just the other day I fixed a script bundled with a hardware system costing around $1M/year to lease to use basename instead of /bin/basename because I wanted to run it, not on RHEL but on Ubuntu and the latter put basename in /usr/bin. Hell, even #!/bin/env bash will break because on another system it's #!/usr/bin/env!! Other scripts did not run because ksh was not installed. At other times scripts break because /bin/sh is (wrongly) assumed to be bash but Ubuntu uses dash, whatever that is.

And this goes from trivial stuff like the above all the way down to gdb being broken by a kernel upgrade to a minor version, or kernel memory leaks caused by minor upgrades, or NFS-related kernel panics.

As to "yet not obsolete" - only if you use the box as a server needing nothing but a CPU, some RAM, an Ethernet connection and some media to boot and serve files from. Try running or compiling an old KDevelop on a new Ubuntu or vice versa and tell me if it's "surprisingly long life yet not obsolete". Also I can assure you that most of my complaints about breakage are met, if the person is a Linux aficionado, with bewilderment at my desire to run such ancient software where "ancient" means "6-12 months old". Certainly the only part of Linux which gives a shit about compatibility is the kernel (you can still run statically linked binaries from 1993), however it doesn't help much because every userspace program outside the very useful but rather limited set of the original Unix tools (grep etc.) happily breaks compatibility at every turn.

I have to say that Windows has far less variants and far less incompatibilities between variants than *nix, so you switch both more smoothly and less often in my experience. This is not to say that it's a lovely experience, just that it's not nearly as terrible as Linux.

Unix-like systems really are relatively stable, in terms of, once you get it working, it will generally continue to work. And, for the majority of superficial commands you might want to run -- ps, top, uptime, w, tcpdump, vi, pico, nano, mail, etc. -- there haven't been many big changes for decades. Somebody that was pretty handy with BSD or Linux in 1995 could still find their way around the commandline on most systems today. They'd have to figure out newer init systems and a few other things, and the popular windowing environments have changed a lot, but basic CLI stuff is pretty stable.

But you're right, trying to get these systems to work in the first place, or trying to keep them up to date without incurring significant cost or downtime, can really be a pain. I've experienced the same. I think the worst offender to date was Samba; at one time, Samba 3 could do file shares with Winbios authentication but not talk ActiveDirectory, and Samba 4 could talk ActiveDirectory but not do file shares (or some such thing, it's been a few years), and they were both being released at the same time, so if you wanted a Samba server in an SBS 2008 environment you had to run both -- on two different systems. And configuring either one of them successfully was an absolute nightmare. I still have a text file under my Documents/sysadmin/ directory that has some shorthand notes on troubleshooting Samba installations and that thing is a gold mine. Every couple of years, after I've forgotten everything about Samba, I get a call to fix one that's not working. I'd never be able to do it without those notes.

That's just one example; I've had lots of issues with Network Manager ("Network Mangler"), print drivers that require 32-bit OS support which, once installed, completely bones other software updates and installations; OS updates that irreversibly break a working Apache installation (that one required me to pull a 24-hour shift building a new server), and yes, trouble with systemd/journald. All kinds of fun stuff.

It's a mixed bag. Windows-based systems usually just work, as long as you throw expensive enough hardware at them. They either do a job or they don't. But they aren't very fixable. There's no advanced logging system in the Windows environment, so whether you have any logs to work with in the event of trouble is entirely up to the individual software vendor, error messages are opaque, and I can't just go grep some source code to try to figure out what the hey is going on. Linux and BSD systems are a lot more fixable, I generally prefer working on them, but they have some of the worst quirky little bugs caused by software affecting other software, and documentation still leaves a lot to be desired.

If one look into a physical toolbox, many of the tools have not changed for generations. Heck, the basic hammer is likely as old as civilization itself.

Thing is that these tools can be combined as the user see fit at time of usage, even it if would freak out the original designer.

Upon writing this i find myself reminded of a Star Trek TNG episode where they have one of the designers of the Enterprise on board, and La Forge is giving her a tour.

She keeps pointing out way they are "doing it wrong" until they have a situation that falls completely outside of design anticipations, and they have to create a solution on the fly.

After that La Forge gives her a explanation about how in the field things rarely work as they do on the drawing board, in particular when millions of light years from the nearest space dock.

As such i think unix-likes are resilient because at its core it was made for admins by admins. It was made to work in the day when you didn't have the net to reach out to for solutions, nor another computer to work on to create the tools to fix what is broken.

One of the horrors of modernity is that we keep forgetting or ignoring how our present lives got bootstrapped in the first place. The kind of A boots B that Boots C that replace A type catch-22.

No one is recommending Linux software to people who don't need or what this level of manipulations.

What I feel is the greatest abusers of these types of over-complications is Windows. I used to volunteer at a library helping elderly people (plus anyone else that needed help). No one came in with a Linux computer.

What the majority of these people came in with were Windows laptops. Laptops that were just too complicated for them to get a hand on.