> The problem isn't so much finding a way of passing metadata but how do you retain compatibility with existing GNU / UNIX tools
Honestly? I'd have suggested a new syscall that provides an interface similar to isatty(). It seems a matter of adding a few constants to the ioctl syscall, prototype it and prove it is useful, then upstream it. Better make a crude prototype, see if that's any useful, and ask upstream what they think of it.
If the new ioctl isn't available, nothing changes, programs continue exchanging whatever they expect. If the new interface is available, most programs won't know about it, and would continue their business as usual. However, programs that know about it could get more metadata trough it, and communicate in a smarter way. Looks like a standard way of introducing new functionality, can be deprecated if needed, and keeps backwards-compatibility, at the expense of a tiny bit of complexity (dwarfed by having to support multiple formats, which some tools already do).
The way you do it seems quite fine too, not sure which would be better.
Staying with traditional file descriptors also has the issues that you are limited to unidirectional communication, and only one file descriptor, so maybe you need a protocol like html, as a sibling joked.
> As for handling graphical components, that's a little easier. Many command line programs already alter their behaviour depending on whether STDOUT is a TTY or not.
Well, that's kind of my point: you could output more than curses interface if you wanted, something like an xml ui file, or a list of commands on a side channel. But I make no judgement on this: it was just to illustrate what kind use-cases this would enable.
So with those points in mind I think this is one of those cases where worse is better. But kludging things a little it means I can have greater ownership of development which means I can actually deliver something that people can use right now. If it gains popularity then maybe I'd have more clout to push through new syscalls, but I certainly wouldn't have that with "only" 500 Github stars :)
> Well, that's kind of my point: you could output more than curses interface if you wanted, something like an xml ui file, or a list of commands on a side channel. But I make no judgement on this: it was just to illustrate what kind use-cases this would enable.
You can do that with my shell right now. The issue is you'd have to write an `open` handler (because how would the shell or terminal know what to do with pseudo-random data if you don't have some kind of handler?) but PNGs, JPEGs, etc will be rendered with sixel/or whatever proprietary support the terminal emulator has for inlining images if `open` is used and STDOUT is a TTY. So it's not just curses interfaces though the principle is similar.
In one experimental module I even went as far as having web pages render in the terminal if STDOUT was a TTY. So instead of making a HTTP(S) request and displaying the bytes, it would render the webpage, images and all. Only falling back to a standard `curl` operation if the command was piped. This never made it to production code though, it was very Linux specific code and required a heap of 3rd party software to be installed just to work. But it was a great example of just how much you can already do with some terminals.
The problem isn't so much finding a way of passing metadata but how do you retain compatibility with existing GNU / UNIX tools? It would be easy to create an entirely new shell (like Powershell) that throws out backwards compatibility but that wouldn't be very practical on Linux/UNIX when so much of it is powered by old POSIX standards.
I addressed this by having a wrapper around pipes. Anything that connects out to a classic executable is a traditional POSIX byte stream with no type data sent. Anything sent to a command that is written to support my shell can make use of said meta data.
As for handling graphical components, that's a little easier. Many command line programs already alter their behaviour depending on whether STDOUT is a TTY or not. I take things a little further and have a media handler function, called `open`, that inlines stuff in a shell that can be (it will render images in the terminal) but if STDOUT is a TTY then it will pipe the image data instead.