But you still have race conditions. The sender would have to ensure that the locks are setup before the receiver calls read() for the first time. Since pipes are often setup by the shell you have no control over the startup times. Sure you could have heuristics such as the receiver waiting a few seconds just in case locks show up, but that just makes things slow and unpredictable.

I stand by my assertion that this can only be solved well (ie 100% predictable behaviour no matter what order things start in or how long they take to intialise) by a new system call/ioctl.

No, I avoid the race condition by: 1) Reader sets the lock before reading any data 2) Writer writes a byte to the pipe 3) Writer waits until pipe is empty (FIONREAD ioctl) 4) Writer checks for existance of lock.

This should be race free.

That requires the first byte sent be compatible with whatever format is ultimately used. As an example for ps, the first byte in JSON should be a { while for plain text it should be space. (We get a little lucky since a space would also be acceptable for JSON, but I doubt there is a universal first byte.) And an initial space isn't accept for a programs like find or grep in either text or null separation mode.

I don't want to belittle what you've done, but the point remains. This can't be done robustly without an additional system call. What you have is tantalizingly close. Even a call as simple as telling the sender that the receiver has called read() would complete your solution.

Not really. All you need to be able to do is to produce the first byte of whatever would have been produced in the "fallback case", i.e. when the reader does not handle format negotiation. Then, when the writer sees that the reader supports format negotiation it will need to signal that the alternative format was chosen. I do this by sending a zero byte (which should never appear in the fallback text format). Then the two first bytes are skipped as part of the negotiation framework when a non-fallback format was chosen.