This is useful because, for example, you stop having the "unmanaged references" you were talking about because now since you are pointing to a version of the data and not the data itself you can be sure of what you are talking about.
It is not a solution for change in time it is a solution for taking change in time out of the equation when we don't need to talk about that. With immutable ds when we talk about data we are just talking about exactly that and time is taken out of the picture. Its called immutable because now you are talking about facts and not the representation in time of those facts. Because now we are talking in versions so it does not make sense. The thing is this data is a snapshot so you are not guaranteed to have the most up to date snapshot. And that's ok because precisely here we take change in time out of the question in order to be able to talk more precisely about the data. Tracking change in time is another story.
So for example you could have things like react. There you have snapshots of the world updating. When you talk about the data it is immutable but then you change it and update the mutable variable where you are keeping change in time.
All of these ideas (and I think they are good ones) are inspired heavily by Rich Hickey's talks and rational behind developing the Clojure language (the author of the post states as much). And while you can use these techniques in other languages/paradigms/problem domains, they are really intended to work well inside the constructs of Clojure, and when applied to "information-driven situated programs" [0] (read business applications with dynamic requirements).
As for some of the short-comings you mentioned:
"But you still need a mechanism to manage mutating data"
Clojure supports this through the use of locking constructs like atoms. [1]
"I think what it's getting at is that you don't really know the precise type of your data, over time, in a distributed system, so it's good to include the flexibility to handle that. That makes sense to me. but generic data structures aren't necessarily always the right way to handle that."
Clojure attempts to bridge the gap between generic data-structures and strongly-typed constructs using run-time specifications. [2]
I mean, the ideas presented here can be generally useful, but your mileage may vary if the principals take you too far out of the idiomatic for your particular language/paradigm/problem domain. If that's the case, you could find yourself wasting energy swimming up stream.
[0] - https://www.youtube.com/watch?v=2V1FtfBDsLU [1] - https://clojure.org/reference/atoms [2] - https://clojure.org/about/spec
I agree. For me it feels like immutable data is a tool, not a universal principle. For example, in low-level C programming or direct control of a register in an embedded context, immutable data isn't a principle, it's just another approach.
What I mean by "universal" is something like SOLID. [1] (Most people only use SOLID for OOP, but Uncle Bob makes it clear in Clean Architecture that he thinks SOLID is universal, and not limited to OOP.)
That said: it does feel like immutable data should be the _default_ approach in many situations. But that's really hard to do in most languages.
[1] https://www.amazon.com/Clean-Architecture-Craftsmans-Softwar...
This is the basic principle behind how sane database transactions work.
ORMs in some languages can be especially dangerous if they overload the getter/setters of the object in such a way that the checkins and checkouts are obscured; you could be passing your object to a function or method that expects to mutate a polymorphic class[0] that is usually a traditional "shared memory" form of objects, well hopefully you can imagine the chaos, redundant database transactions, consistency problems, failure modes, uncaught exceptions, etc. that are going to be a nightmare to debug.
[0] worse yet, imagine if it's someone else's code and they change the api from not mutating to mutating for performance reasons. Will you notice the documentation change or the changelog? It's bad enough in the case when it's not an ORM and just a mutable object.
These are are part of a book that is being written at the moment as far as I can tell:
"This article is an excerpt from my upcoming book about Data Oriented Programming. The book will be published by Manning, once it is completed (hopefully in 2021). "
The rise and fall of paradigms that present themselves as panaceas is instructive. You have "structured programming", "object oriented programming", "functional programming" and now "data oriented programming".
What I'd like to see is paradigms paired with "where this works well" rather than paradigms sold based on "this will solve the software crisis", "this works better(unqualified by when)" and "if you're not doing this, you're doing it wrong". The later two claims leave a bad taste in the mouth of the casual users/observers, who gradually morph into active critics and sink the paradigm, wasting the good and useful parts of each (OOP has a vast universe of haters because it's most successful of the panacea-paradigms so far and as a panacea, there's much to hate here but still).
SQL is about as data-oriented as it gets, you have a programming model that's constrained and focused on data layout, structure and performance over being a general purpose language. There's a good article a while back about ECS that I can't find which talks about how most performant ECS start to mirror SQL and other row based data engines.
So the parent is correct. It's not enough to say data orientation is a good thing. It needs to be compared with previous approaches (encapsulation) and then explain what the tradeoffs are.
You still use if/then/else and repetition and subfunctions in OOP. But you restrict and replace some of their previous usage.
Same for functional programming: you take OOP but remove all the sideeffects. You still have objects which encapsulate data which can only be accessed in a certain way and can behave polymorphic. Mind that inheritance was never really part of OOP in general.
Now about DOP - not sure what exactly it adds on top of FP or if it is even an established and well defined term. Doesn't look to me like it, looks more of a bundle of recommendations right now.
Data Oriented Programmming is both old and new. In that it does not have the same amount of programming patterns that OOP has. As it is a more bare metal means of programming without a ton of abstraction to ease most programmers into it.
Where I find the idea interesting is concurrent and parallel processes are more natural in the data oriented. And that is through immutablility and ownership as first principles.
Mutability is just a hack to save some memory.
.. and time. If we needed to compute an account balance by summing all the debits and credits since the account was opened...
Traditionally people use mutable struct or objects for non-primitive data structures, which implicitly allocate addresses in computer memories - that is a very implementation-wised approach, more specifically, a very Von Neumann approach.
For example, there are many systems having entities in the databases, like Person { id, name }. But if there's no database, in many programming languages people would write Person { name } only. If people write the in-memory version first, they have to re-write their models. Systems with mutable references inconsistently force the referencing system on people by default, instead of just giving people only they ask for.
In modern days, a lot of computing goes beyond a single computer, there are many distributed systems, a lot of serialization is going on, and the reference in one computer's memory doesn't mean anything to another.
If we try to loosen the definition of the computer, we can think of many systems as bizarre computers. If we take a look at data processing systems like Spark or Flink, conceptually they also look like computers - bunches of hardware and 'operating systems' sit on top of them, but in the form of clusters instead of single computers. In such cases, there are no shared memories, and the memories are implementation details - users won't aware of them like they have to realize there are underlying memory systems in Von Neumann computers. In such kind of systems, people only care about computing which is more substitution based instead of mutable references. While at most, they only need to aware of nodes as a whole in the cluster.
It's close to the 'substitution model' vs the 'environment model', or the 'functional' vs the 'object-oriented' described in the SICP. Where I found conceptually the 'substitution model' is more fundamental, and simpler - it's basically elementary or middle school maths.
> The problem here is that data isn't actually immutable (generally) and mutability isn't actually the problem. The problem is unmanaged references or other dependencies on the mutable data.
So another novel way to look at it, is if you don't have mutable references, you don't have to manage it, then the problem is eliminated as a whole. Like Haskell, Erlang, or Spark as we just mentioned. At the end of the day, you can have references in outer worlds, be it ST monads, processes, database, or just our real world, sometimes the processing part of the program don't have to care about them.
The big one: "Data is immutable". The problem here is that data isn't actually immutable (generally) and mutability isn't actually the problem. The problem is unmanaged references or other dependencies on the mutable data. The "source of truth" becomes muddled which creates the problem of how to keep the various instances in sync (or otherwise handle cases where they are/become out-of-sync). Immutability is a very useful tool since you have have any kind of dependency -- direct, indirect, implied, etc) -- and there's no worry. But you still need a mechanism to manage mutating data. Maybe it comes out somewhere, but the principles of DO don't cover it, which is a rather serious omission.
Also, principle 2 isn't really a principle. I think what it's getting at is that you don't really know the precise type of your data, over time, in a distributed system, so it's good to include the flexibility to handle that. That makes sense to me. but generic data structures aren't necessarily always the right way to handle that.