Re:Software reuse is more like an organ transplant than snap...
by mike_ivanov on Wednesday June 24, 2020 @05:58PM
(#23634340)
Yes, the structure is always there. The trick with unix pipes is that interpreting the structure is always on the receiving side. I don't care if this is CSV or TSV if all I want is the number of lines. I don't care about concrete representations such as JSON or YAML if all I want is to change one string (like a hardcoded server name) to another (like a variable reference). Can you see that? How about this: why would one need a spec or schema in order to pipe an XML file through gzip?
Re:Software reuse is more like an organ transplant than snap...
by fiddlerwoaroof on Wednesday June 24, 2020 @09:10PM
(#23635763)
Yeah, I think people underestimate the value of an “escape hatch” when dealing with structured data: a Java-style API can be great at controlling downstream developers and easing library maintenance. But, as a “end-developer” such APIs can be absolutely miserable to discover when you’re trying to quickly pull out a bit of data.
Re:Software reuse is more like an organ transplant than snap...
by shadowgovt on Wednesday June 24, 2020 @11:00PM
(#23636445)
But pretending JSON or YAML is something you can safely "change one string" on is a recipe for disaster unless you have total control of the whole pipe chain, input to output.
Re:Software reuse is more like an organ transplant than snap...
by mike_ivanov on Thursday June 25, 2020 @01:00PM
(#23642790)
How is it less safe than changing them in a (scriptable) text editor?
Re:Software reuse is more like an organ transplant than snap...
by shadowgovt on Thursday June 25, 2020 @01:40PM
(#23643292)
It's not. Neither is a safe approach. A safe approach to mutating JSON is something like a small amount of JS to read in the JSON, mutate it as javascript objects, and JSON.stringify the result.
Re:Software reuse is more like an organ transplant than snap...
by shadowgovt on Wednesday June 24, 2020 @11:08PM
(#23636488)
Good point. Perhaps it is more accurate to say that with pipes, the structure is purely up to correct semantic interpretation to maintain, whereas other abstractions carry structure alongside the payload as part of the spec of the abstraction.
Of course, one can always botch configuring that structure. You never really quite escape the rule of "garbage in, garbage out."
You used to be able to cat things to /dev/dsp, but- that used something like 8khz, 8-bit, mono audio. That's horrendous. Because, with just a stream of bytes, you have to settle for the least common denominator- /dev/dsp had IOCTLs to set sample rate, number of channels, and bit depth, but... with just a stream of bytes, you can't do that.
Similarly, video data via /dev/fb0 - AFAIK you don't even have defaults there to rely on, to display anything useful you need to do IOCTLs to find out about its format.
When do you not want structure? Seriously-
Plain, human-readable ASCII text is maybe a candidate - but even then there's implicit structure (things like handling CR/LF, tabs...)
Unicode text? You know that's structure. (Ever had a read() call return in the middle of a UTF-8 multi-byte sequence?
CSV? That's structure.
Tab-separated columns? That's structure.
Fixed-width columns? Also structure.
You don't get to not have structure. Structure is always there. The question is whether you get to have a spec for your structure, or whether it's just "well, from the output it looks like column 3 here is always the hostname of the server I want, so I'll use 'cut' or 'awk' to extract it". That approach can work in practice, but...