Copy & paste is the basic, ubiquitous, doesn't-try-to-do-too-much IPC mechanism that allows normal users to shovel data from one program into another. As simple as it is, it's indispensable, and it's difficult to imagine trying to use a computer without this feature.

The same applies to pipes, even though they work a little bit differently and are useful in slightly different situations. They're the "I just need to do this one thing" IPC mechanism for slightly more technical users.

Remember the first iPhone? I wasn't into it, but a bunch of my (senior developer) colleagues were. I asked them how they lived without copy & paste and they all told me it was just no big deal.

I would guess the reason you dont do it much on iphone is the lack of multitasking support.

Android alt tabbing has always been a basic part of life and now split screening is a thing too.

You lot getting widgets soon, maybe you'll get alt tab next year?

I understand it's a "hidden gesture" and thus all the UX complaints apply, but from pure observation most android users use the app switcher about as often as PC users use alt+tab... basically not as many as you think.

Most mobile phone users who are even slight technophobes will switch apps the old way from the home screen or app drawer. at least with the iPhone the "going home" behavior can accidentally surface the app switcher which after enough exposure might train such users to use it.

Idk, I kinda hate all the UX paradigms that android introduced, it often felt like they ignored much of what xerox parc taught as objectively good UX and tried to go it on their own, only to have to relearn all the same mistakes that Mac and Windows learned over the last 3 decades.

(ignoring everyone young enough to not remember that iphone first many years didn't have copy paste)

The iphone NEVER replaced a computer.

It replaced the television. No need for copy paste there.

The ipad is today trying, very shy, to hint at replacing a couple small use cases of the computer.

> Are we all not glad we don’t use the Unix method of communicating on the web? Right? Any arbitrary command string can be the argument list for your program, and any arbitrary set of characters can come out the other end. Let’s all write parsers.

The way that I think about this is that the Unix philosophy, which this behavior is undoubtedly representative of, is at one end of a spectrum, with something like strict typing at the other end. Rich, being a big proponent of what is described in the article as "Lego-like" development clearly does not prefer either end of the spectrum, but something in-between. In my opinion as well, the future of software development is somewhere in the middle of this spectrum, although exactly where the line should be drawn is a matter of trade-offs, not absolute best and worst. My estimation is that seasoned developers who have worked in many languages and in a variety of circumstances have all internalized this.

And yet, at least for the Unix tools I've used, nobody did write an elaborate parser. Instead, they all ended up using newlines to represent a sequence. There were never really nested structures at all. That least-common-denominator format they were forced into ended up making input and output really easy to quickly understand.

Maybe the problem with Rest APIs is that JSON does too good of a job at making it easy to represent complex structures? Maybe we'd be better off using CSV as our data format for everything.

Say Unix tools returned JSON. What good is that without a schema? Instead of a parser, let's all write schema transforms?

sed and awk are the elaborate parsers that are used in complex unix command line scripting.

The biggest mistake in the design of Linux is that they allow newlines in filenames. Seriously, what the heck?!

Um, but that's exactly what we do use on the web. Oh, sure, there's some really popular formats for input and output with widely available parsers (HTML, XML, JSON), and HTTP itself (as well as other protocols) specifies headers and other formats that need to be parsed before you get to those body formats, including telling you what those headers tell you about the other formats so you can no which parsers you need to write, or use if you can find one written for you.

A lot of the value of scripting, pipes and associated concepts is low barrier to entry.

I agree with the value of strong typing. But I also remember how infuriating it was to learn to work with a type system when I was learning to write code.

When I need a quick answer of some sort, iteratively piping crap at a series of "| grep | awk" is exactly what the doctor ordered. Sure, I could bullet proof it nicely and make it reusable by investing time the time to write it in something saner, but there's zero reason to - I'm not likely to ever want to perform the same action again.

Unfortunately this is complexity in and of itself. I don't disagree that different cases require different tools, however the split should be strongly weighted in one direction. Mostly IPC over pipes with a few exceptions, mostly REST and JSON with a few exceptions, mostly language X with a few exceptions. Everyone will have their own preferences, but I think it's important to pick a side (or at least mostly pick a side) or else you accept chaos

- At the scale of 1,000 to 1M lines of code, authors/programmers have some control, and you're dealing with a time scale of months or years.

- At the scale of 10M to 100M lines of code, no organization is an author with complete control, and you're dealing with a time scale of decades.

In the latter case, you can't make global changes to the system, so you end up with glue in the form of pipes and textual data dumps.

You're going to have some chaos because you can't go and change every system to be consistent.

So I would say the future of software development is at BOTH ends of the spectrum, not somewhere in the middle.

-----

Good material on the subject: https://www.dreamsongs.com/Files/DesignBeyondHumanAbilitiesS...

If you look at the code in a web browser or operating system you will see this "lack of control". Once a codebase reaches a certain size, there's only so much you can do with it, and you end up with loosely coupled glue around it.

I think a reasonably minimal Debian system has to have somewhere between 100M and 1B lines of code in it, and that whole thing becomes a blob that you dump on a virtually free computer like a Raspberry Pi, etc.

I think if you look at the software that banks and airlines run on you will see the same thing. There's some code from the 60's or 70's written in some weird language at the heart of it. You don't really get to architect the system; it's more accurate to say that the system places its constraints on you.

Uzbl is a collection of "web interface tools" that adhere to the Unix philosophy, that come together to create a browser.

https://www.uzbl.org/

He clearly uses a different web to the one I use. >.>

You could almost say that micro-services are trying to follow in the Unix tradition. But the problem is that a) they don't really get used in that ideal, small-scale use-case because they're almost always written and consumed internally, not exposed to the public, and b) they do get used in those huge, complex cases where their lego-ness stops being a virtue and starts being a liability.

SaaS is all the rage now, and the business of SaaS is disincentivized from making "lego-like" software. The easier it is to interoperate, arguably the easier it is to replace or reimplement your service.

Because a lot of the value of unix tools is derived from their interoperability, they play nice with each other.

An interesting inverse-example is Slack, which does lots of interop with other services, because it isn't a trillion-dollar company that can attempt to give you everything you need under a single roof.

Right, just like LEGOS thrive in simple cases, are not suited for building complex systems.

That is why the idea of "Software LEGOs" is not reality. Or you can say that Unix Pipes are "software LEGOs", with similar limitations.

In my practice I have used both with great success. For logging, parsing, and displaying playback data from field systems, UNIX and UNIX-like tools have been incredible. VNLog in particular is a wonderful way to interact with data if you need just a bit of structure on top of unix outputs.

And anyway, getting from not-so-great data to something that a typed library in language X can parse is a great job for plain old Unix tools.

I recently picked up Swift and it was a dream. Never used it in nearly the same degree, but I was highly productive quickly.

The thing is: I don’t think it’s a bad paradigm (everything a stream of text), it’s just the “IDE” sucks completely. If there was inline autocomplete, peek-ahead results, a “verbose mode” where the options are all more like typed full-English objects, instant fuzzy search of actions, multi line with an easy way to edit, etc, it would probably be fantastic and I’d be a 100x shell developer. It would also basically feel like an IDE at that point for a normal programming language!

1) data is a stream of bytes

2) data is only a stream of bytes

That's it. And it turns out that's a pretty powerful abstraction... Except it requires the developer to write a lot of code to massage the data entering and/or leaving the pipe if either end of it thinks "stream of bytes" means something different. In the broad-and-flat space where it's most useful---text manipulation---it works great because every tool agrees what text is (kind of... Pipe some Unicode into something that only understands only ASCII and you're going to have a lousy day). When we get outside that space?

So while, on the one hand, it allows a series of processes to go from a text file to your audio hardware (neat!), on the other hand, it allows you to accidentally pipe /dev/random directly into your audio hardware, which, here's hoping you don't have your headphones turned all the way up.

This example also kind of handwaves something, in that you touched on it directly but called it a feature, not a bug: pipes are almost always the wrong tool if you do want structure. They're too flexible. It's way the wrong API for anything where you cannot afford to have any mistakes, because unless you include something in the pipe chain to sanity-check your data, who knows what comes out the other end?

You used to be able to cat things to /dev/dsp, but- that used something like 8khz, 8-bit, mono audio. That's horrendous. Because, with just a stream of bytes, you have to settle for the least common denominator- /dev/dsp had IOCTLs to set sample rate, number of channels, and bit depth, but... with just a stream of bytes, you can't do that.

Similarly, video data via /dev/fb0 - AFAIK you don't even have defaults there to rely on, to display anything useful you need to do IOCTLs to find out about its format.

When do you not want structure? Seriously-

Plain, human-readable ASCII text is maybe a candidate - but even then there's implicit structure (things like handling CR/LF, tabs...)

Unicode text? You know that's structure. (Ever had a read() call return in the middle of a UTF-8 multi-byte sequence?

CSV? That's structure.

Tab-separated columns? That's structure.

Fixed-width columns? Also structure.

You don't get to not have structure. Structure is always there. The question is whether you get to have a spec for your structure, or whether it's just "well, from the output it looks like column 3 here is always the hostname of the server I want, so I'll use 'cut' or 'awk' to extract it". That approach can work in practice, but...

Of course, one can always botch configuring that structure. You never really quite escape the rule of "garbage in, garbage out."

But that's not what an abstraction is! UNIX pipes are maximally low-level, as close to un-abstract as it is possible to get, except perhaps if it used bit-streams instead of byte-streams. It literally cannot get less abstract than that.

UNIX pipes are completely untyped, like a C codebase that uses only void* to pass all data. (Okay, some way of indicating end-of-stream is also needed, but it's still a good analogy.)

> pipes are almost always the wrong tool if you do want structure

Not almost the wrong tool, entirely the wrong tool.

You can't magically shoehorn types back into an untyped system when all the existing components assume untyped streams.

PowerShell's structured and typed object streams is more UNIX than UNIX: https://news.ycombinator.com/item?id=23423650

If this were true, then a typed system couldn't exist on top of machine code and random access memory, because the existing components (i.e. CPU + random access memory) have no knowledge of the types you are speaking of.

I think the assumption is false. Types can be shoehorned onto untyped data.

To put a spin on it, maximally low-level within it's particular domain, perhaps, sitting on top of the transistors below them, sitting on top of quantum physics.

Thinking a bit further, an abstraction appears to be the mapping in both directions between two domains. That being said, the two domains themselves, are independent, and don't appear to be the abstraction.

The verb "abstract" would mean we are mapping from one domain to the other.

I think "homomorphism" is the mathematical term for what I'm describing.

What's my point? We can always go lower. :)

I agree, and it just hit me while reading your comment that the special thing is not just that you can plug any program into any other program. It's that if one program doesn't work cleanly with another, this enforced simplicity means that you can easily modify the output of one program to work with another program. Unix command-line programs aren't always directly composable but they're adaptable in a way that other interfaces aren't.

It's not great for infrastructure, don't get me wrong. This isn't nuts and bolts. It's putty. But often putty is all you need to funnel the flow of information this one time.

This is why functional programming and lisps are such fantastic development environment, because you can use components (functions) that are not very opinionated about what they are acting on.

BTW if I was making UNIX command line today it would use LinkedHashMaps for everything instead of text streams.

Consider

    cat myData | toolStripsWhitespace > myData

    var myData = RestApi->read(myDataRecordID)
    var mutatedData = someMutation(myData)
    RestApi->update(myDataRecordID, mutatedData)

    var myData = new SomeObject(RestApi->read(myDataRecordID))
            ->someMutation('params')
            ->someOtherMutation('params')

    RestApi->update(myDataRecordID, myData->toJSON())

    var myData = RestApi->get('/records', id=myDataRecordID)
    myData = RestApi->post('/tools/strips-whitespace', myData)

    cat myData | stripWhitespace | allToUppercase | removeTheLetterC > myData

    var myData RestAPI->get'/records', id=myDataRecordID)
    myData = RestAPI->post('/tools/strips-whitespace', myData)
    myData = RestAPI->post('/tools/allToUppercase', myData)
    myData = RestAPI->post('/tools/removeTheLetterC', myData)

Of course in the real world, you can put whatever endpoints you want into your API, and there are dozens of wildly different APIs being called RESTful, but regardless of any of that, rarely are they conducive to being used in pipes, because the point the GP was getting at is that tools are made in the UNIX ecosystem with the pipeline in mind, and that is not the case with REST endpoints.

Thanks for the discussion by the way!

I do think that there is an interesting point around "designed with pipelines in mind" though. I don't know if that would specifically make things more "lego-like", or if that would even specifically be a good thing. But it would be certainly interesting to analyze the list of "average library interface, or framework, or domain-specific language, or REST API" and evaluate each of them along those lines, and consider what it would look like if each was designed them to be more pipeline-centric.

Or maybe the original post came more from a feeling of "it's easy for me to think in terms of pipelines, so anything that forces me to do otherwise feels needlessly complicated"?

But, if we start thinking in terms of pipelines, then that's essentially what functional programming is, run a function, get this output, pass it to another function, get its output, run another function, get its output. So, maybe the love for pipelining comes out of actually a desire for a functional core. So it'd be more like "i only want to think in terms of functions at the core and not have to worry about side effects, and unix does this on a program level".

So, of course, you can't entirely get away from side effects, otherwise you'd literally be doing nothing. So we push it to the edges as far as we can. The analogy for side effects for Unix then would be `> myData` as the last step. So you could say that Unix's philosophy is to have programs that form a functional core, with there being only one possible side effect, which then only has one possible placement, which is at the end of everything. Sounds like a functional dream world.

Back to the list, if we wanted to design "average library interface" with this idea, then maybe you'd say here is a package with all the functional methods. And here is a package with all the side effect methods. And the design is to run a bunch of methods from the functional package before finally making a call to the side effect package.

If we then loop all the way back to REST APIs (and I mean REST spec this time) then I think I could make an argument that REST does in fact follow this functional core philosophy. You have GET endpoints which are your functions, that let you retrieve resources. And then you can POST against objects, which are your side effects, which let you create or update a specific resource. The spec limits the types of side effects you can make, almost to the point of making them dead simple. So, we then have a clear delineation between what is a function and what is a side effect, with a very limited and clear number of side effects that you're able to create.

If you ever wondered why some people are obsessed with functional programming this is the reason why:

Functional programming forces every primitive in your program to be a Lego Block.

A lot of functional programmers don't see the big picture. They see a sort of elegance with the functional style, they like the immutability but they can't explain the practical significance to the uninitiated.

Functional Programming is the answer to the question that has plagued me as a programmer for years. How do I organize my program in such a way that it becomes endlessly re-useable from a practical standpoint?

Functional programming transforms organ transplantation into lego building blocks.

The "lego" is the "function" and "connecting two lego blocks" is "function composition".

In short, another name for "Point free style" programming is "Lego building block style" programming.

There's no reason you can't do this with functional programming, but obviously you can do it with non-functional programming too, and you could certainly fail to do this with functional programming.

You're not free from the need to specify how to parse the strings just because your whole human readable system is stringly typed.

It comes with the same drawbacks (the lack of static typing is similar to strings with Unix; you have to be aware of the details of the data coming in and out because there is no type system to save you - unlike Haskell), but the benefit is easier composition.

In either case you have to compose functions With the correct types otherwise you hit an error. The difference with dynamic types and static types is when this error occurs. Run time for dynamic types and compilation time for static types. Either way some type of error will always occur regardless.

Though I agree with you if you have a limited number of types things become easier to deal with.

But being able to specify interfaces for lambdas is absolutely fantastic. Something like: "this function takes an object and returns a function that can take either a string or a number and returns a number" is easy to do in Typescript, and useful for figuring out where the lego block fits.

I.e. this will warn (inc "hello") in clj-kondo

Good interfaces enable unexpected things.

https://en.m.wikipedia.org/wiki/FUDI

Obviously not, the types have to be compatible just like how a lego piece must be compatible.

>The programmer has to have the foresight to make their types compatible.

Just like a lego builder needs to have the foresight to see whether two lego components are compatible. The analogy still fits.

> I think the novelty of the Unix pipe for interoperability is that they (typically) work on one agreed-upon kind of data: human readable, white-space separated. So a lot of tools "just work" with each other.

The agreed upon data is just a type. Unix pipes compose functions that take in strings as input and deliver strings as output.

The spaces is just an oversight, the whole thing would have been more clean if the type was designed to represent multiple pieces of data like a tuple of words rather than a single string with words divided by spaces.

>you could certainly fail to do this with functional programming.

Just like how two lego pieces can't compose if they're not compatible pieces facing the right direction. Sockets and plugs must be compatible. A functional program still fits the analogy of legos perfectly.

Take a look at lego pieces here: https://brickarchitect.com/2019/2019-most-common-lego-parts/

There exists lego parts that can never compose with certain other parts. Just like functions and function composition.

What you're not seeing is how other styles of programming such as OOP or procedural programming fail to fit the analogy and literally become like organ transplantation.

Let's take a look at object composition. How does that work in the context of legos? It's like a lego block with a mutating hole in it and once you can put different things in that hole and that changes the overall lego block.

Also with OOP and procedural programming you get lego blocks that can mutate. Lego blocks can transform themselves and other lego blocks. They form an interconnected network of entities that are constantly mutating.

To fit the analogy of a lego block you need unchanging blocks.

The entire field of Procedural programming and OOP is basically the art of building systems with mutating primitives that constantly change. Imagine constructing buildings using bricks that change shape. Now imagine refactoring code that does this.... Organ transplantation.

Also you really just need to try it. Doing the type of grafting and refactoring that's an intrinsic part of programming literally becomes lego-like once you program using the functional style. To really see the big picture I suggest you exclusively try the point free style using Haskell.

That being said. There are still massive tradeoffs to doing functional programming. But if your goal is to program like you're using lego blocks, functional programming is the path.

That kind of sounds like a structure of living cells. They're more complex than a structure of lego blocks, but they can do a lot more too.

Don't take that as an argument in favor of OOP though. Building with legos is a lot easier to get right than building with cells would be.

The key lies in your choice of primitives. Do you start off with a set primitives that are extremely complex or a set primitives that are simple but complete in the sense that you can build entities of unlimited complexity by composing simpler primitives?

I would say the latter method is easier. But there's no reason why the former method won't work. Biological evolution has built many examples of working machines with primitives of unimaginable complexity.

Basically, it's the infinite monkey theorem.

Pure functional programming is complex, one has to compose effectful functions differently to pure functions. But the pieces do really fit like Lego bricks, especially when the same mathematical abstractions are used consistently. The Haskell community has been extremely effective at creating such consistency, by promoting various abstractions using category theory as a guide. The object-oriented community is not so different in this regard with their promotion of "patterns".

I've been writing Haskell professionally for 8 years. The problems with Haskell are the tooling, language stability, the learning curve and the difficulty of reasoning about performance, especially space usage. But composition and re-use works.

The key ingredient in making software reusable and portable is the talent and experience of the engineer, regardless of language.

If it's an abstract data type, it doesn't have to be understood by the caller, it just has to be passed to another "lego brick" which understands the abstract interface. If it's a structured data-type, then there's no reason why it shouldn't be understood by the caller, the type should tell you how to consume it. I do not understand your point.

> So the solution is to make that data easily convertible or generic.. and guess what? That's no different than doing the same in an OO language

I guess you mean structured data types here? But you have missed my point completely about side-effects, it is side-effects (coupling via back-channels) that prevent composition, in general, in an OO language. OO languages also typically have an obsession with nominal types that can impede reuse.

> The key ingredient in making software reusable and portable is the talent and experience of the engineer, regardless of language.

You appear to be suggesting that languages/tools don't matter? Why not aspire towards languages that encourage safe composition and re-usable software? Your argument reduces down to "good motorcyclists don't need helmets".

> If it's an abstract data type, it doesn't have to be understood by the caller, it just has to be passed to another "lego brick" which understands the abstract interface. If it's a structured data-type, then there's no reason why it shouldn't be understood by the caller, the type should tell you how to consume it.

And this is exactly how any well constructed OO API works as well.

You can have really good and really bad APIs in any language, it really is up to the skill of the developers. I firmly believe that, there’s no language or tool that suddenly makes you create better things.

I have in my jobs interacted with high profile robust APIs in C++ as well as functional languages. They can be a joy to use when designed well, regardless of language.

Well it depends on the type of composition you're talking about. Anything when mashed together hard enough can compose.

I recommend you try building something complex with the point free style using Haskell. It will be a bit mind bending at first but if you get it then you'll see how FP is basically the same thing as using legos to build data pipelines... exactly the style used with unix pipes.

Only in the same sense that all computing is Turing Machines or NAND gates.

This is a very common misunderstanding, but there is a reason that FP, which is transformational, had to adopt dataflow in order to sort-of handle reactive systems.

Functions run to completion and return their result. Filters tend to run concurrently and, importantly, do not return their results, they pass them on to the next filter in the pipeline.

It is possible to compose any two filters. It is not possible to compose any two functions, not even close, the default for functions is to not compose (arity, parameter types, return type,...).

Filters are functions. They are one and the same. For unix it's a function that takes in a string and outputs a string. The unix world reduces everything into a singular type. You will note that unix "filters" cannot compose with functions of "other" types either. If unix included types like arrays or ints in stdout you would have the same typing problems as you have with functions.

In the unix world all your types are strings so things seem simpler, but in reality your little unix programs need to deserialize the strings into proper types in order to do anything meaningful. You could achieve the same thing if you used functions that returned strings all the time then did internal deserialization but that's not really a solution.

I do get your point about arity and parameter types. You're referring to the fact that not all functions can compose UNLESS they have compatible types and an arity of 1 in the parameters and an arity of 1 in the return value (Golang for example can have arity > 1 in the return value).

However this arity issue doesn't really exist. A function with Arity of two is isomorphic to a function of arity 1 that takes in a 2-tuple as input.

  f(a, b) -> c
  f'(Tuple[a, b]) -> c

  g(d) -> Tuple[a, b]

The only compatibility problems with composition among functions in the end is just types (not arity) but this problem still exists even in the unix world... it's just hidden from you because you don't actually watch the programs perform deserialization and serialization.

As I explained before: no they are not.

They have some similarities, but they also have the important differences I outlined above.

The single type is also a restriction relative to the functional model, and crucial to (syntactic) composability.

> If unix included types like arrays or ints in stdout ...

But it doesn't...

> you would have the same typing problems as you have with functions.

...so you don't.

> Both f' and f are equivalent in theory.

But they are not actually equal (never mind the same), at least not in typed FP, and most FP is typed.

> it's just hidden from you

Exactly. It is hidden in Unix and not in FP. That's a difference.

> You could achieve the same thing if you used functions that returned strings

Two points:

(1) you write that you "could" achieve the same thing. Meaning that, once again, they actually are not the same. Otherwise the "could" wouldn't make sense (and it does).

(2) returning strings would not be the same at all, because filters do not "return" their results. They return status codes. The result is not returned, it is passed directly to the next filter, continuously.

So: it is good to recognize that 'X is similar to Y', but that doesn't imply that 'X is just Y'.

Let me address this point so it is more clear. Functions by definition do not need to "run" at all. The underlying implementation of a programming language may "run" a function lazily, eagerly or concurrently but that is not what the Definition of a function is describing.

A function is simply a relation between two sets, one set called a codomain, another set called a domain is deterministic. That is all.

So within unix piping the domain is stdin, the codomain is stdout, the function is the unix program. This paradigm fits the definition of a function and as long as that "function" remains deterministic and compose-able it fits within the paradigm of functional programming BY definition. there is no need to discuss "similarities" here, because it fits the definition therefore it is functional programming. Of course we can get pedantic about certain non-functional use cases but in basically you are fully aware of the generality I'm describing. No point in getting pedantic.

Also keep in mind when you "execute" these functions it does not matter HOW these functions are executing whether concurrently, lazily or eagerly. The only important thing is the relation between the domain and codomain. That is all.

>The single type is also a restriction relative to the functional model, and crucial to (syntactic) composability.

Every unix program must internally deserialize this string type and serialize it again for output. The typing must be handled regardless. It seems like you don't need to deal with it but it's dealt with regardless. You also get inevitable deserialization problems of incorrect types. You do not escape typing in the unix world.

>But they are not actually equal (never mind the same), at least not in typed FP, and most FP is typed.

A relation that takes a string from a domain and returns a string that is from a codomain is a function that is part of functional programming. This is EXACTLY what a unix program does when relating stdin to stdout and therefore it IS a function and it is FP By definition.

You can look at it from another direction. A function that returns an Array does not fit the definition of what a unix program should output to stdout therefore a function IS NOT a unix program. BUT a unix program IS a function.

Formally unix programs are just the set of all functions where the domain and codomains are strings.

>>Both f' and f are equivalent in theory. >But they are not actually equal (never mind the same), at least not in typed FP, and most FP is typed.

The mathematical term is "isomorphic" meaning you get the EXACT same properties going one way vs. the other way. It's the syntactic sugar that prevents composition. Using multi-arity functions and single arity functions with tuples only involves two additional parenthesis:

   f(a, b)
   f'((a, b))

Arity and tuple types are well described to be isomorphic in category theory. I'm not making this up.

One way to think about an isomorphism is that if two things are isomorphic then they are just different perspectives of the exact same thing. Any differences are superficial or illusory.

>Exactly. It is hidden in Unix and not in FP. That's a difference.

Exactly what? If you're not dealing with the type conversions someone else has too. Just because it's hidden from you doesn't mean it doesn't need to be dealt with. Someone has to serialize things and deserialize it. Even from the command line level you cannot escape types. Let's say you have a unix program that can only take numerical strings from stdin. Let's say you pipe english letters to it... if you do then you will have still triggered a sort of type error despite everything being typed as strings in unix. It's just not handled explicitly as a system error.

You cannot escape typing even in the unix world because it is in the end just FP.

>So: it is good to recognize that 'X is similar to Y', but that doesn't imply that 'X is just Y'.

Except we're not talking about similarity. Like I said above X is just Y BY definition.

Not sure what else I can do to make you understand this.

As such, I am bowing out as I can only repeat what I’ve written.

Here's what I did: I defined what a function is, then I described how a unix program fits the definition of what a function is...

In short a unix program with stdin as domain and stdout as a codomain is a function but a function is not necessarily a unix program.

I think what's actually going on is you didn't read what I wrote very carefully. You sort of just skimmed over it. I can't blame you, it is rather long and detailed...

But if you want to have a meaningful discussion you need to read it and ask questions about things you don't understand.

> > Unix piping is basically functional programming.

> Only in the same sense that all computing is Turing Machines or NAND gates.

Yes, you can map and analogize long enough until you reach a point where you find what you perceive to be equivalences.

However, what you've done at that point is rediscover that all these mechanisms are computational and thus at some level equivalent and transformable into each other, just like yoo can implement all of this with just NAND gates.

You have not shown what you claim. ¯\_(ツ)_/¯

https://qbix.com/platform

Reusability on the web. Here is where we are going:

https://qbix.com/QBUX/whitepaper.html#Distributed-Operating-...

Except you litterally use side effects to communicate. Not really FP, that part.

In a pipeline like

seq 100 | number | grep '[a-z]' | sort | tr -d '.' | tr a-z A-Z

every component is a pure function (byte strings -> byte strings) and the pipeline's results are totally deterministic [if you don't change the locale or collation order with environment variables]. No component of the pipeline makes any change to the filesystem¹.

¹ although both stat(2) and inotify(7) mechanisms could allow other processes to detect the interactions with the filesystem that are caused by running the pipeline, among other process-related mechanisms that system as a whole could use to detect how many times the pipeline has been run, so Unix itself is definitely not side-effect-free for almost any operation

Or any other state change.

Because it's not stateless.

Hence not FP.

Clearly not all command lines you can type in the shell are stateless (reading the disk, network, stdin, clock, or system RNG, among other things, isn't, and none of those things are rare or hard to do) and the shell doesn't specifically encourage you to write stateless code and it doesn't have a formal way to verify whether or not a particular command like is stateless or not. All of these properties are pretty similar to most popular FP language interpreters except for the last two. I guess it is a relevant distinction that your FP interpreter would typically know (as a matter of static inference) whether an expression is stateless, while your shell would typically not know.

If you want to get pedantic fine, but the majority of use cases is stateless.

Probably most complex shell pipelines follow this pattern in practice, but the first counterexample I thought of was shuf, or sort -R. Also, many pipelines that include any kind of interpolation or variable substitution also don't follow it.

A different insight about reasoning about pipelines might be that the byte string (or ASCII string) type is too weak to catch most kinds of errors, and it's not uncommon for one pipeline component to not, in fact, be a total function with respect to the actual data type that you're trying to capture (I don't know the right terminology for this). This most famously happens when you do regular expression substitutions but your regular expression doesn't actually match the full grammar that you're looking for. Then the pipeline can be incorrect as a whole for some inputs, but earlier and later stages don't notice. It can also happen anywhere that different tools have a different implicit understanding of the relevant grammar or structure.

That connects up with all sorts of other ideas which are actually about type safety and parsing more than FP. For example, PowerShell has tried to take the pipeline concept in a different direction, to the consternation of us Unix purists. Its use of typed objects in this context makes more explicit what the contract between programs in the pipeline is supposed to be. There is also a LANGSEC connection in terms of the risks of informal or underspecified parsers and grammars.

I know I've personally written lots of Unix pipelines that were correct for all the inputs that I personally threw at them, but definitely not correct for every possible input. I like to use ! in vim frequently to shell out to a command line to perform a text editing task, and vim has an associated undo and redo which means that sometimes I'm trying several variants until I find the one that successfully appears to do the edit that I intended. Sometimes I do this at an almost preconscious level, which is really strange in terms of thinking about the concept of the correctness of a program (in this case, where the program is literally only going to be used once, with the programmer looking over its shoulder).

On the other hand, if you're trying to build something large (a house, say - a real house, not a toy one), you don't want the lego interface. Sure, you can plug anything to anything, but not all of those connections make sense. Also, for building something that large, legos are too small a building block to be convenient. You want some larger things - beams and sheets of wood and particleboard, pipes, air ducts. You don't want to have to build all of those out of tiny blocks.

In the same way, I wonder if building larger applications out of FP is going to be similarly tedious. I have never done it, so I will admit that I don't know.