foster_nyman
Categorizing Variants of Goodhart's Law
One way to translate your exogenous/endogenous split into his language is: runaway happens when the “selection function” gets trapped inside the system it’s selecting, so the feedback loop selects for its own reinforcement rather than for wider viability. Sexual selection is the clean biological example because preferences can become an internal amplifier: once a trait becomes a strong signal, the preference and the trait can co-evolve into something locally stable but globally costly (tails, antlers, etc.).
Where Bateson gets especially sharp is on schismogenesis (i.e., interaction patterns that escalate because each side’s behavior becomes the stimulus for more of the same). In that sense, a lot of ethno-nationalist / eugenic thinking looks like an attempt to institutionalize a narrowing selection function (“select for X”), while simultaneously insulating it from the broader ecology of feedback (social, economic, moral, informational) that would normally check it. That’s how you get stable local maxima that are brittle and, often, destructive.
On exogenous selectors, though, I’d be cautious. External selectors can absolutely be less confounded by local identity incentives, but they also bring their own blind spots (mis-specified metrics, distance from consequences, Goodhart effects). Bateson’s recurring warning is basically that the more you collapse your evaluation to a single axis, the easier it is to optimize yourself into a corner.
If you want a specific thing to watch for as you read Ecology, keep an eye on how often he treats “pathology” as a property of relationships and feedback, not of individuals. That’s the bridge between mating preferences, ideology, and organizational dynamics.
Structural Stability and Morphogenesis: An Outline of a General Theory of Models [pdf]
How Nature Works: The Science of Self-Organized Criticality [pdf]
https://ejcj.orfaleacenter.ucsb.edu/wp-content/uploads/2017/...
"All biological and evolving systems (i.e., individual organisms, animal and human societies, ecosystems, and the like) consist of complex cybernetic networks, and all such systems share certain formal characteristics. Each system contains subsystems which are potentially regenerative, i.e., which would go into exponential "runaway" if uncorrected. (Examples of such regenerative components are Malthusian characteristics of population, schismogenic changes of personal interaction, armaments races, etc.) The regenerative potentialities of such subsystems are typically kept in check by various sorts of governing loops to achieve "steady state." Such systems are "conservative" in the sense that they tend to conserve the truth of propositions about the values of their component variables—especially they conserve the values of those variables which otherwise would show exponential change. Such systems are homeostatic, i.e., the effects of small changes of input will be negated and the steady state maintained by reversible adjustment."
Green Corn Ceremony
[edit]: found it - https://news.ycombinator.com/item?id=46595199
Schismogenesis
Silent House Typology [pdf]
I read Graham’s point as narrower than “there’s nothing to learn.” He explicitly says: “There are certainly some political questions that have definite answers.”
The warning label is about identity capture. Once a view becomes part of who you are, the odds of real updating drop: “people can never have a fruitful argument about something that’s part of their identity.” Or, put positively: “you can have a fruitful discussion … so long as you exclude people who respond from identity.”
So the issue isn’t the topic. It’s what happens when belief turns into a kind of badge.
For me, at least, this is one of his most important essays and worth re-visiting from time to time - https://paulgraham.com/identity.html
"I think what religion and politics have in common is that they become part of people's identity, and people can never have a fruitful argument about something that's part of their identity. By definition they're partisan."
Steps to an Ecology of Mind [pdf]
Macy Conferences
Chesterton's Fence
Coase Theorem
What is the Point of Equality? [pdf]
Seeing Like a State: How Certain Schemes to Improve the Human Condition Failed [pdf]
What I’d love to see (maybe as a follow-up) is the structural question: what policies reliably prevent the “service creep” phase from eating the research phase? I suspect the answer is boring and contractual, which is exactly why it might matter.