The design was "strip away as much from Multics to get it running on this crappy computer". That after a few iterations this might generate a philosophy of composition is another matter entirely. And I'm not really arguing against that, at least for a very small subsection of Unix' timeline (around v5-ish?). But the assumption that this was all a masterstroke of, erm, intelligent design seems a bit outlandish.
It seems to have been an element of Plan 9's design, though.
And let's not get into "decades of experience" and "empirically". There's way too few data points and way too much simple economics muddying those waters. By the same argument I could say that Unix really failed, as the biggest applications (nay, the systems themselves) on today's variants are quite monolithic and/or isolated subsystems themselves (browsers, languages, most servers).
The design ideas originated with Unix. They were quite explicit, read this book: http://www.catb.org/esr/writings/taoup/html/index.html
The main design idea behind UNIX was trimming MULTICS to fit into a PDP-11 with a new architecture. More effective composition was already demonstrated by Dijkstra's THE, Burroughs B5000, and even MULTICS. Their method, an unsafe language + heavyweight composition, led to a system that was both enabled for composition and quite unreliable/insecure. Later, Wirth pulled on pre-UNIX work to show how much better it could be done with Modula-2 etc. Nastiness of system hid away, type/interface checks, and otherwise just programming modules calling functions in modules. That's composition done right.
It's no surprise that best technologies around improving reliability, maintenance, throughput, etc always involved working around rather than with UNIX model.
" That means it survives, which we have observed empirically."
Empirically, we've observed COBOL on System/360 survived with other mainframe systems. As did Windows monolithic kernels and apps. DOS was all over the place. What survived tells you more about economics and sociology than whether an approach is technically superior.
"Lisp is not a better Unix."
http://www.symbolics-dks.com/Genera-why-1.htm
Yet, UNIX still can't do all of the above despite matching or exceeding many items on list through labor put in. Those rarely crashed, had OS + apps in same language, took you right to OS code with OS-related error, and let you patch live systems to keep them running. On top of LISP advantages like incremental, per-function compilation. I'm a Linux user that doesn't code LISP. Even I still see that UNIX + C architecture can't touch LISP architectures in flexibility and safety combination. And, no, that's not a simpler problem at all given how many PhD's have tried, failed, or only approximated such results in UNIX + C. It's just a simpler, consistent, designed-in approach to solving hard problems instead of scaling up a platform that chopped down ALGOL and MULTICS to fit on a PDP-11 plus some arbitrary changes.
I really don't know where this ideal model of UNIX lives.
All your "not true under Windows" can be extended to all other OSes that aren't UNIX.
Windows isn't the only non UNIX OS, although it appears to be like that to most HN eyes.
Whenever something hits the real world, people will add hacks to solve their immediate problems. That's just a property of software that has nothing to do with Unix.
When you have good bones, the hacks have something to rest on. The system still has some coherence, and doesn't degrade into a pile of mud. Linux is wading dangerously into that territory.
When something is widely used, there is no option to not have hacks... you either have bad architecture and hacks, or good architecture and hacks.
There was a thread here a few days ago discussing how NT is the same... it has good bones from Cutler, but then Microsoft had to layer all the compatibility hacks on top, for Windows 95 earlier. That's probably true to some extent.
I guess I'm mainly saying that Windows is inferior in terms of system administration and scripting: shell scripts, fork/exec, shebang lines, the file system abstraction, etc. Threads and async I/O were not in the original Unix model, so it's perhaps not surprising if Windows does that better.
If AT&T was allowed to sell it, I doubt very much the market would have seen such good bones.
So like in any free gift regardless of quality, people build gadgets on top of gadgets and think that is the best there is.
The UNIX haters handbook still holds quite some truths when checking an actual version of HP-UX or other commercial UNIX es.
A fact of the UNIX free culture is that people only started contributing to gcc after Sun decided to stop offering the developer tools.
Some of us prefer doing administration of machines via tools like Active Directory or have an OS that has security access lists applied to every kernel object.
Also UNIX has lots to learn from mainframe OSes like IBM I, z/OS or Unisys MCP.
But their owners sells them, they weren't forced to give the source code for free to universities.
As long as it's possible to build these systems efficiently on top of the kernel, that means the kernel succeeded. ioctl() is the hack where the kernel APIs failed.
However, there was an excellent paper on HN a few weeks ago: https://news.ycombinator.com/item?id=11652609
Their observations about IPC are great. Android, OS X, and Ubuntu/Linux have their own IPC abstractions with structured data. So arguably for this class of applications, Unix APIs failed. (But note that Android is Linux but not Unix, because it's basically a single language OS, which uses single language IPC.)
"We observe across all three OSes that no POSIX calls belonging to traditional POSIX IPC are among the frequently invoked operations. As discussed earlier, Android departs from the traditional IPC in favor of higher-level IPC abstractions, namely Binder, the core Android message passing system. Similarly, OS X supports higher-level IPC primitives built atop Mach IPC, which diverged from POSIX since its inception. Finally, Ubuntu also provides applications with message passing capabilities based on D-Bus."
I'm not sure how well-supported this is in the historical record.
You give /proc as an example, but it's first appearance was in the 80s, and it took until solaris in the early/mid 90s to really get going. ( https://blogs.oracle.com/eschrock/entry/the_power_of_proc )
Type compatibility enables composition. That is, given functions `foo :: A -> B` and `bar :: B -> C`, you can form the composite `bar . foo :: A -> C`. Unix provides a very simplistic solution that guarantees type compatibility (all I/O is byte stream I/O) at the price of creating other problems (what should be the result of `lseek()`ing a socket?).
I noted in my message that Unix polymorphism isn't perfect, and comes with the costs of serialization and escaping. However, I challenge anyone to design something better with types. It's not like people haven't tried.
From Alan Perlis: "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures."
That is the essence of Unix (and Lisp for that matter). You don't want A, B, and C; you want just A, and then 100 functions on it that can be composed.
Who said anything about monads? I only established the conditions for composition to be possible. Composition is the binary operation of a category, and, while every monoid is a category in a very trivial way, there are categories with more interesting structure. But, categorical blablabla aside, what matters is that making everything a byte stream isn't the only way to make things composable.
> From Alan Perlis: "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures."
Well, Perlis is wrong on this one. It is better to have types and operations that reduce the number of ways in which your program can possibly fail.
It would be a mistake to bake types into the OS -- whatever that even means -- because nobody has found the perfect type system. For every type system, there are some things that don't fit in the model. (This reminds me of some theorems in the foundations of mathematics...)
There are even libraries to shell scripts in Haskell and OCaml. If there were some inherent advantage to doing so, they could easily replace shell scripts. But I predict they will not.
I don't do kernel programming, so I don't care what kernels do internally. As an application programmer, I want usable APIs. I want an API that makes resource leaks inexpressible, that doesn't let me seek intrinsically sequential streams, etc. Of course, such an API can't possibly be a C API.
> Haskell and OCaml run on Unix just fine.
Haskell has an I/O model that's barely safer than C's. The only improvements are that the `Handle` type is abstract and the `bracket` function is a `try...finally` of sorts. OCaml fares even worse: Can I open a single file for both input and output with just the standard library? In either language, is there a comprehensive solution for manipulating memory-mapped files?
> It would be a mistake to bake types into the OS -- whatever that even means -- because nobody has found the perfect type system.
Making a usable interface to OS services doesn't require a “perfect” type system, just a type system that prevents the most common mistakes that happen when using OS services. OS services are to a large extent about manipulating resources, so substructural types can help a lot.
Composition shows up in the following ways:
1) Everything is a file -- open() / close() / read() / write() are polymorphic, with ioctl() and others for the necessary hacks that don't fit the model. Devices are files; sockets are files; pipes are files; /proc is a file system and not a bunch of C APIs. Compare with Windows. This is not to say there's no cost to this uniformity. There is -- serialization and escaping bugs -- but in practice the benefit seems to have outweighed the cost. You can't underestimate composition, because without it your code collapses under its own weight.)
2) In addition to files composing, file systems compose. mount() is a way of composing a uniform namespace from disparate entities. (Not true under Windows)
3) Processes compose. You can do stuff like "sudo chroot ls ...". fork/exec allows one process to set state for another, without code changes to the child. (Also not true under Windows)
4) Language heterogeneity. Via the simple mechanism of the shebang line, shell can invoke awk/sed/Python/Perl/Ruby/JS in the same way. And the shell itself. (You could call this an instance of the Composite Design Pattern).
Unix is certainly messy NOW, but that's because it is so widely used that everyone needed to add their own stuff to it (threads and shared libraries being very popular and arguably anti-Unix). You can try to make something "cleaner", but it will likely collapse under its own weight. There is no uniform model that encompasses all of computing. It's all little truths in specific situations. That's why there are so many little languages.
I've had this argument before on HN: Lisp is not a better Unix. Lisp is solving a simpler problem; decades of experience have shown that. Unix is solving a harder problem, doing it in a somewhat simplistic and strained fashion, but it's the best we know how to do right now. Lisp or Smalltalk don't solve the same problem by any stretch of the imagination.