The generation number idea you have shown is an excellent idea. It immediately enables several new capabilities compared with just mutation:
* You are now able to determine which data is older and which is newer without resorting to an external clock.
* If your mutation takes a long time, there is no longer a need to use a long-held mutex to ensure thread safety which would prevent concurrent reads; you can create a new generation separately and then atomically CAS it. Writers don't block readers.
* If you suddenly need undo/redo functionality, it's suddenly trivial. Or diff'ing between versions. Or understanding the reason for changes (basics of auditing).
Thus from a practical perspective the foundations of actual computing must be built on mutable state.
You are wrong about the generations idea. It's a side effect. Functions themselves cannot actually do anything with a generation number. It's completely useless other then for letting you know how many times a value has changed.
A generation number also doesn't negate the need for mutexes. The system is still operating on shared state, a generation number doesn't change this.
Yes, theoretically. Now imagine your mutable state is 2GB in size, have fun creating a copy of it on every change.
And all this is heat rather than light because there is absolutely a significant fraction of concerns that both sides would agree are utterly incidental/accidental complexity.
what is important is that we understand which definitions people were using in particular utterances, that our definitions have adequate ontological coherence (avoiding "eargrayish" reasoning errors), and that other people can understand which definitions we are using
in brooks's 01986 paper he clearly considers questions like copying 2 gigabytes of data 'accidental' http://worrydream.com/refs/Brooks-NoSilverBullet.pdf
> All software construction involves essential tasks, the fashioning of the complex conceptual structures that compose the abstract software entity, and accidental tasks, the representation of these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints. Most of the big past gains in software productivity have come from removing artificial barriers that have made the accidental tasks inordinately hard, such as severe hardware constraints, awkward programming languages, lack of machine time. ... The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract, in that the conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed. ...
so it isn't important what mike acton thinks the terms should mean unless we're trying to understand how he uses them; brooks, moseley, and marks lump space and speed constraints into the category of 'accidental' rather than 'essential', and their arguments need to be interpreted using the definitions they intended to use, not conflicting definitions invented by a youtuber decades later
similarly, people commenting on moseley and marks's paper should be assumed to be using the same definitions as moseley and marks, not conflicting definitions, unless they explicitly state otherwise
like, lots of programs are specified to do one or another thing with the filesystem, and have to include extra complexity to do it, but the filesystem is something we introduced and could do without; it doesn't exist in objective reality outside the computer. is that complexity accidental or essential? at the level of the program it's essential (especially if the program is something like find(1) or cp(1), whose job can't be defined at all without presupposing a filesystem) but at the level of the system it's accidental
Division of labor: making accidental complexity essential since 1945.
Then again, perhaps we need to account for opportunity cost. Even if persistent storage can take many forms, it's hard to imagine that equivalent features could be 10x simpler. Maybe Chuck Moore would disagree, but his modus is usually to insist you don't need something.. (also totally fair)
In the event you find the source code TLDR, indeed it seems implementing an immutable api is complex.
That is essentially what must be going on under the hood to model mutable state. Every mutation made to a variable is a commit. The only commits that are saved are the ones that have existing references.
Now you're thinking it could be a big memory leak. But you pair this with reference counting you can eliminate the leak.
Essentially when the reference of the earliest commit goes out of scope, the next earliest commit is checked out as the initial root commit and the current commit is freed.
The memory model is then like a moving git window/linked list. Mutations are committed at the tail end while memory is freed at the head.
All of your asterisked points are well-taken, but: do you need that capability? Sometimes; sometimes not.
However, that modeling introduces complexity of its own that not everyone agrees is entirely essential.
The cost is efficiency but make no mistake not thinking about the heap allows people to build much more complex programs. It's a trade off between complexity and efficiency.
Mutable state does not do this. Your idea of generations doesn't make sense, because that means every function must take a generation as the input parameter too. Given a different generation number the function must deterministically create the same output.
It is not a complete system. Your equations produce this useless generation number that aren't reused as part of the system, it's just there for you and meta analysis.
That is not to say your first paragraph is completely wrong though. There is a trade off between modularity and efficiency. Mutating a variable is less resource intensive then generating a new variable.
Mathematical purity is indeed what allows you to treat your logic like bricks and building blocks. My point is that "a monad is a monoid in the category of endofunctors" is not the only "pure math" out there, and also that your domain model is likely the most impure part of your whole program. Functional programming is awesome! But mostly ignores the existence of essential mutable state, and embracing functional programming is only a small part of the "mathematical purity" struggle that is indeed the only way to build truly reusable building blocks. If you're spending lots of time building clean, pure, mathematical data manipulation logic, but the data you're manipulating is "Dog : Animal" and "Cat : Animal", you're in a garbage in/garbage out situation. Worry about the mathematical purity of your data model itself. It will marry and dance and harmonize with the purity of your functional logic!
> Mutating a variable is less resource intensive then generating a new variable.
Not always.
Arguably most of it!
After all, computer don't compute.
https://www.youtube.com/watch?v=EKWGGDXe5MA&t=297s
One of the miseries of life is that everyone names everything a litte bit wrong, and so it makes everything a little harder to understand in the world than it would be if it were named differently. A computer does not primarily compute in the sense of doing arithmetic. Strange. Although they call them computers, that's not what they primarily do. They primarily are filing systems. People in the computer business say they're not really computers, they are "data handlers". All right. That's nice. Data handlers would have been a better name because it gives a better idea of the idea of a filing system.
Recursion programs are provably Turing equivalent to WHILE or GOTO programs, but that doesn't mean they are equally natural for our brain to think about. (Not to mention things like tail recursion, which is even less intuitive.)
I can believe that recursion is even less easy to teach than iteration, but it may be that this is a pedagological issue, or that we have yet to work out the best way to convey or document computation.
That's easy mode. Mutability is often justifiable when one person is doing one thing to a computer at a time. Now extrapolate to multiple people editing the same document at once. Suddenly you're discussing CRDTs and other approaches. And now implement undo on top of that!
Likewise with git, or blockchain, or kafka. They're persistent logs so you can keep your head straight while figuring out what the current state(s) of the system can be. Even with git, when you do an in-place mutation (force-push) there's still an extra log (reflog) behind the scenes trying to keep the changes sane.
Regarding, functional programming and mutation. In a typed "pure-FP" language like Haskell: - you need to explicitly declare what constitutes a valid value - which drives what valid values are observable between program steps - which informs how corrupted a data-structure can be in your program
For instance, using a tree-shaped structure like `Tree1 = Leaf Int32 | Node Int32 Tree1 Tree1` and you have some `MutableRef (Tree1)`. You know exactly that from the ref you can read a whole Tree1, you can change the content of the MutableRef but you need to give a whole new Tree1. In particular, you are sure to never read "an half-initialized tree" or "a tree that was changed while I was reading it" because these things just do not exist in the realm of valid Tree1s. Such simple guarantees are really important to many programs but are not guaranteed in most programming languages.
Of course, if for some reason (e.g., for performance) you need something more flexible and are willing to pay the extra complexity, you can do it. As an illustration, you can augment your Tree1 with some MutableRef as well. `Tree2 = Leaf Int32 | Node Int32 Tree2 Tree2 | Cell (MutableRef Tree1)`. Here Tree2 contains mutable references to a valid Tree1 so Tree2 is not fractally complicated but you could do that too. With Tree2, you can interleave reads, writes, and other operations while working on a Tree2. These extra effects could be required (for performance reasons for instance) at the expense of inviting surprising behaviors (bugs). In pure-FP it is clear that with Tree2 we lost the possibility to "read/write a whole tree in a single program step". Thus, if the control flow of the code is not linear (e.g., multi-threaded, callback-heavy) you may have fun things occurring, like printing a half of the tree at a given time and the second half of the tree after a mutation, resulting in printing a tree that never existed. Enforcing global properties like the heap-invariant becomes inconvenient because it is actually hard if we let mutation in. I'd go as far as saying that the Tree2 doesn't exist as a tree: given a Tree2 we are merely capable of enumerating chunks with tree-shapes piecewise.
This reply is long already, so I won't go into how Haskell has various flavors of mutable refs in IO, ST, STM and recently even got linear-types to allow some fine-grained composition-of-atomic-mutations/interleaving/prevention of effects. But overall I find pure-FP takes mutability much more seriously than other more mainstream languages. Here, pure-FP just keeps us honest about where non-determinism bites. The collective failure to realize is one of the key reason why we are in The Tar-Pit: violated in-memory-invariants can compound even outside programs by forcing outside/future users to deal with things like misleading information being shown to users, urgently upgrading your fleet of servers, days of one-off data-cleanup and so on and so forth.
Functional programming is only "mathematically pure" because we already have math for it. You can make mutating state "mathematically pure" too, if you just invent new math for it. For example, you can pair state with a "generation" variable and define mutating functions as returning the new state with generation + 1; now all your functions are pure again. That's not all it takes, and it's not easy! But it shows how narrow-minded the current idea of "purity" is.