Past a certain level of complexity, that's no longer true.
_Seeing Like a State_ is a great introduction to this, but I think Carol Sanford's work goes much more into detail. The main thing with the high-modernist view that James Scott was critiquing is that it comes from what Sanford would call the Machine World View. This is where the entire system can be understood by how all of its parts interact. This view breaks down at a certain level of complexity, of which James Scott's book is rife with examples.
Sanford then proposes a worldview she calls the Living Systems World View. Such a system is capable of self-healing and regenerating (such as ecologies, watersheds, communities, polities), and changing on its own. In such a system, you don't affect changes by using direct actions like you do with machines. You use indirect actions.
Kubernetes is a great example. If you're trying to map how everything work together, it can become very complex. I've met smart people who have trouble grasping just how Horizontal Pod Autoscaling works, let alone understand its operational characteristics in live environments. Furthermore, it can be disconcerting to be troubleshooting something and then have the HPA reverse changes you are trying to make ... if you are viewing this through the Machine World View. But viewed through Living Systems World View, it bears many similarities to cultivating a garden. Every living thing is going to grow on its own, and you cannot control for every single variable or conditions.
For similar ideas (which I won't go into detail), there is Christopher Alexander's ideas on Living Architecture. He is a building architect that greatly influenced how people think about Object Oriented Programming (http://www.patternlanguage.com/archive/ieee.html) and Human-Computer Interface design (what the startup world uses to great affect in product design).
Another is the Cynefin framework (https://en.wikipedia.org/wiki/Cynefin_framework). Cynefin identifies different domains -- Simple, Complicated, Complex, and Chaos. Engineers are used to working in the Complicated domain, but when the level of complexity phase-shifts into the Complex domain, the strategies and ways of problem-solving that engineers are used to, will no longer work. This includes clinging to the idea that for any given problem, there is a global solution which will satisfy all local conditions.
The funny part about this speech is that he's just telling everyone they did it wrong, and Richard Gabriel agrees:
https://dreamsongs.com/Files/DoingItWrong.pdf
The point of his pattern language is to enable people to create their own architecture for their own needs. The point of OOP design patterns is to lock you in a prison of enterprise Java programming in the Kingdom of Nouns. Of course, I think everyone realized this like a decade ago.
The parent comment wasn't stating this. It was stating that there could be a partial global solution that would benefit all microservices, a solution which teams would have to adapt for covering local conditions as well. A middle ground per se.
Thanks for sharing the "Living Systems World View" btw, very interesting!
Just for the context: I would say I'm a natural intuitive bottom-upper, except that I can't help but reconsider everything my intuitive self learns from a strongly analytical top-down way.
From that perspective and 30+ years of experience (where I like to think I'm at least open to being completely wrong about anything and everything), I think top-down, prescriptive solutions can be useful and effective, but need to understand and carve out the holes (and conservatively large ones at that) for "local" concerns - BTW, "local" often typically just means lower, where there the lower level itself can have "global" and "local" concerns.
Now, I know this often doesn't happen, so let's lay out how it can work:
- there's a top-down person -- "Tim" in the article -- who has responsibility for for developing a solution
- there are the separate teams, who are responsible for communicating feedback on potential solutions.
Also, I wish I didn't need to point this out, but "responsibility for" === "authority/control over".
(If that's not the case, then never mind: you essentially have a "free-for-all" organization, and just better hope someone who's not too crazy wins the cage-match and can hang on long enough to be a net positive.)
If team X doesn't know Kafka, then they can't tell you the ways in which it's not as good as their existing (potentially ad-hoc, but definitely working!) message system. There may be things that their system does that the team just automatically assumes all message-brokers will do because it's "obvious" that it's needed.
If, on the other hand, someone on team X organically considers Kafka as a local solution, learns it, tries it out, all of this stuff becomes obvious immediately.
So the pure top-down approach has two possible solutions:
1. It gets useless feedback "meh, seems fine"
2. All N organizations actually take the time to try out the solution before giving feedback, which means you spend a lot of resources evaluating each top-down hypotheses
The suggested solution from TFA is to have a top-down person embed in one team, find some improvements that work locally, then embed in a second team and do the same. Only then should one try to generalize from the shared experiences of the team. It recognizes that good feedback is expensive and bad feedback is likely, so just cut out the whole "give feedback" stage and have the top-down person learn from actually being embedded in a couple of teams.
Ha, ha! A concept for suckers.
GIGO does not imply its opposite.
>> That's just flat wrong. A global solution can solve global concerns and also allow for local conditions.
So lets rephrase that. A global solution that ignores local conditions will have problems and will likely fail.
Suggestions are usually well grounded (e.g., "let's migrate to this `std` class instead of this old home-rolled wrapper), but sometimes there's some nuance to how something is currently done and deep discussion of the proposal can work through these bits.
In theory, yes. But in practice, no (and this is the author's point, I think). In theory, the more "local conditions" you have to account for, the more exponentially complex your "global solution" becomes. (This is the "state" metaphor.) In practice, you can't build that impossibly complex system (and it might not be desirable, anyways!) - so you're likely to try to change local practices in service of a more streamlined global solution. The more you do that, the farther away you move from respecting local conditions.
If there’s one Tim per team, you’ll have 100 Tims proposing different global improvements and 100 teams needing to respond intelligently to those suggestions.
Not if standardization is the priority.
Is standardization always a priority?
Caveat, for small enough problems, good enough solutions, and charismatic enough leaders, global solutions can work. But they all break eventually.
But I would stress less that "Seeing Like a State" -- that is a top-down, global solution -- was not the problem.
The problem was that "Tim" didn't really understand the problem he was trying to solve (well, none of us truly understand very much at all, but he didn't understand it better than many of the teams associated with the individual services).
"Tim"'s proposal probably solved some problems but created various other problems.
The best solution, though, (IMO) isn't that Tim should be smarted and better informed than everyone else combined, nor that every team should continue to create an independent solution. Instead "Tim" could propose a solution, and the 100 micro service teams would be tasked with responding constructively. Iterations would ensue. You still really, really need "Tim", though, because multiple teams, even sincere and proficient ones, will not arrive at a coherent solution without leadership/direction.
> A global solution, by necessity, has to ignore local conditions.
That's just flat wrong. A global solution can solve global concerns and also allow for local conditions.