Or you could read short book on game theory, think for ten minutes about perverse incentives, and get to the same results without the conceptual diarrhea that Yudkowsky insists on. Why he had to try to drag a concept of free energy into it with no justification, I don't know.

"Read a short book on..." Is a pretty high bar for conceptual simplicity.

Why wouldn't those who have had exposure to the concept of free energy prefer that metaphor to yours? You may be biased having exposure to more game theory than free energy. One person's conceptual diarrhea is another person's cleanest explanation.

>"Read a short book on..." Is a pretty high bar for conceptual simplicity.

Take five minutes and read the wikipedia page on the prisoner's dilemma then.

Even in a two player game, it's clear why individually optimizing for personal gain leads to poor outcomes for everyone, yet if there was a way to force people to act in a particular way, the total outcome for everyone would be better for all.

Except people tend to solve the prisoner's dilemma problem well enough that many research experiments lead to cooperative strategies. So it's a poor model of the real world. It's a dilemma for people who believe everyone's a rational optimizer.. for lack of a better word let's call them economists. There are variations of coordination problems that are actually complex enough to go unsolved in the real world and in some cases they don't even necessarily require irrational decisions, they just require a large amount of coordination to adjust the state to something where each individual decision has justified payoff.

Except that the one forcing people will inevitably force people to act in a way that maximizes his/her personal gain.

Which is fine if that is also a way which maximizes those peoples' personal gain as well.

But is rarely or never the case. See pretty much every dictatorship in history.