-Z, --null
Output a zero byte (the ASCII NUL character) instead of the character that normally
follows a file name. For example, grep -lZ outputs a zero byte after each file name
instead of the usual newline. This option makes the output unambiguous, even in the
presence of file names containing unusual characters like newlines. This option can be
used with commands like find -print0, perl -0, sort -z, and xargs -0 to process
arbitrary file names, even those that contain newline characters.
-z, --null-data
Treat the input as a set of lines, each terminated by a zero byte (the ASCII NUL
character) instead of a newline. Like the -Z or --null option, this option can be used
with commands like sort -z to process arbitrary file names.
man xargs: --null
-0 Input items are terminated by a null character instead of by whitespace, and the quotes
and backslash are not special (every character is taken literally). Disables the end of
file string, which is treated like any other argument. Useful when input items might
contain white space, quote marks, or backslashes. The GNU find -print0 option produces
input suitable for this mode.
man find: -print0
True; print the full file name on the standard output, followed by a null character
(instead of the newline character that -print uses). This allows file names that con‐
tain newlines or other types of white space to be correctly interpreted by programs that
process the find output. This option corresponds to the -0 option of xargs.And yes, I would expect that find detects that when it is talking to xargs then null termination should be used without the user having to go and fish out what the options are for each tool. And if you used ps with another tool that prefers json then ps can automatically do that, again without having to find and maintain flags.
content nagotiation only works with bi-directional data transfer (i.e. not with pipes).
It uses file locks (F_SETLK) on the pipe with a magic offset value offset to do the negotiation.
I stand by my assertion that this can only be solved well (ie 100% predictable behaviour no matter what order things start in or how long they take to intialise) by a new system call/ioctl.
This should be race free.
I don't want to belittle what you've done, but the point remains. This can't be done robustly without an additional system call. What you have is tantalizingly close. Even a call as simple as telling the sender that the receiver has called read() would complete your solution.
We've already seen something like this - for example ls does column output if going directly to a screen, otherwise one per line, and many tools will output in colour if applicable. However this is enabled by isatty() which uses system calls, and inspecting the terminal environment for colour support.
Another example is telnet which does feature negotiations if the other end is a telnet daemon, otherwise just acts as a "dumb" network connection. (By default the server end initiates the negotiations.)
However the only way I can see this being possible with pipes is with kernel/syscall support. It would provide a way for either side to indicate support for richer formats, and let them know if that is mutually agreeable, otherwise default to compatible plain old text. For example an ioctl could list formats supported. A recipient would supply a list before the first read() call. The sender would then get that list and make a choice before the first write() call. (This is somewhat similar to how clipboards work.)
So the question becomes would we be happy with a new kernel call in order to support rich pipes, which automatically use current standard behaviour in its absence or when talking to non-rich enabled tools?
I would love it if grep/find/xargs automatically knew about null terminating.