De facto is a far cry from "required by the OS". You'll notice that the programs we both mentioned (as well as programs that interpret whitespace this way) tend to operate on human-readable text, which uses whitespace to separate words. However, there are many other programs that do not operate on human-readable text, and do not rely on whitespace to delimit records in a pipe (or other file-like construct). Thus, the OS should not try to enforce a One True Record Delimiter, since there isn't one.
> ...and suggest that perhaps there are additional conventions, that if sufficiently simple, could be used to build composable programs that don't need to understand a format that is particular to just one program.
This really is the heart of the matter. There is a trade-off between the specificity of the conventions and the freedom of the program to interpret the data however it wants. UNIX is at (almost) the extreme right end of this spectrum--the only convention it imposes is that information must be represented as 8-bit bytes.
My question to those who feel that UNIX is too far to the right on this spectrum is, what are some conventions that can be adopted universally that won't break composability? I'm not convinced that there are any. Even simple things like requiring programs to communicate via a set of untyped key/value pairs (where each key and value is a string of bytes) would be risky, since it could easily lead to the creation of disjoint sets of programs which only work with members of their own sets (e.g. members of each set would require set-specific key/value pairs).
produce-data | munge | invoke-seperate-programming-language | munge | do-something-with-data
I mean we have to do the C->Bash->Awk->Bash->C precisely because nothing actually works together and thats with just text. If we did mandate some IPC and ran across a program that couldn't speak it? Well as long as our smart program had a toString equivalent we wouldn't be any worse off than we are now. But between programs that spoke it we'd be better off.You seem to contradict yourself :) Yes, you might have to munge some text to get the disparate programs to work together. But, at the end of the day, you get the result you want, because munging the data is possible. Contrast this to a world where munging the data is all but impossible, since each program speaks different objects. Then you'll have some real breakage on your hands, unless you can write O(N^2) inter-object mungers to replace the O(1) text-based mungers you were using before.
> If we did mandate some IPC and ran across a program that couldn't speak it? Well as long as our smart program had a toString equivalent we wouldn't be any worse off than we are now. But between programs that spoke it we'd be better off.
How would this be an improvement? Most programs (i.e. the ones that don't know your "smart" program's IPC object format) would fall back to using the toString() method. The comparatively small set of programs that can use your program's IPC objects would be tightly coupled to each other, meaning a change to the IPC object structure or semantics will require modifications to most/all of the programs. If anything, your proposal exacerbates the "disjoint sets of programs" problem, and has the effect of turning the legacy/compatibility option (toString()) into the ironically future-proof method for interacting with "smart" programs.
Don't get me wrong, I won't turn down structured IPC just because it's structured :) However, I have yet to see an IPC system that (1) gives you more than a byte-stream, and (2) allows for universal composability. Every IPC system I have ever come across sacrifices one of these properties to make gains in the other. While I haven't ruled it out, I'm skeptical (but would love to be proven wrong) that there exists a form of IPC that can meet both requirements.
In the very near future, everything on Linux will be done with dbus.
Or the output of a 'uniq -c', you can then 'sort -g' to order lines by the number of occurrence, but if you want to take the top 5 lines and discard the counts with 'cut', you need whitespace transformations in between. (AWK would be an alternative to cut that performs the whitespace conversions on its own, but AWK is a full blown programming language, so one may as well have something that deals with dictionaries, arrays, etc, as input types anyway).
All this is to say that the untyped bytestream relies on conventions in Unix to make the bytestream useable between composable programs. These conventions are adequate for current uses, but show some weaknesses, and suggest that perhaps there are additional conventions, that if sufficiently simple, could be used to build composable programs that don't need to understand a format that is particular to just one program.
I totally agree that objects (meaning data + associated data-specific code) are probably overkill, though optional object interfaces may be nice if the programmer is willing to pay the computational cost, like one does when using AWK over cut. And I definitely think that inheritance is an idea best to avoid for data.