News for hackers. Stuff that matters.
Re:Trapped priors as a basic problem of rationality by philipkglass on Saturday March 13, 2021 @02:37PM (#26449271)
Rationality in service to what end, though? You can't just build some complicated instruments to record measurements of the ideal polity, the way you can refine measurements of the electron's mass. Most political disagreements are about values more than wonkish policymaking.

But in fact many political zealots never accept reality. It's not just that they're inherently skeptical of what the other party says. It's that even when something is proven beyond a shadow of a doubt, they still won't believe it.

This could describe cognitive error [1] or it could describe someone who's using deontological reasoning more than consequentialist reasoning. (And perhaps lacking the vocabulary to say "consequentialist arguments won't sway my values.")

It is probably a good strategy to refuse to be talked into positions against your fundamental values by mere evidence. I'm joking but also serious. I have a vague sense that torturing suspects doesn't work to prevent terrorist attacks, but that's not the root of my opposition to torture. Maybe clever torturers could make it work more often than not. That wouldn't sway me, because torturing suspects is against my values. Documenting how well it can "work" won't make me reconsider.

[1] Further food for thought: "cognitive error" and "revealed preference" may also be different names for the same thing. Are they experimentally distinguishable?

Re:Trapped priors as a basic problem of rationality by renewiltord on Saturday March 13, 2021 @02:59PM (#26449459)
Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately.

As for the rest, yes, I agree with you. Often, after having considered a great deal of evidence, I condense the finding into some strength of belief about something. Now, absent the original evidence I have a position. This is usually fine since I actually lack the ability to summon all evidence on demand.

It's okay, though, since I (and I suspect most people intuitively) have both a likelihood notion and a certainty in likelihood notion analogous to a Confidence Interval and P-value, if you will.

So I might hold the belief that GME is going to be between 120 and infinity on Jan 13 2022 and have a certainty in that belief that is like 10% but lose the evidence that made me think that.

This is also sort of a failure mode for fast Aumann Agreement even in co-incentivized individuals - the fact that memory is short.

So the problem is that my certainty holding is not well done.

Thanks for the thought prompt.

Re:Trapped priors as a basic problem of rationality by astrange on Saturday March 13, 2021 @04:20PM (#26450102)
I've always found it odd that the people called "rationalists" are not rationalist. Talking about evidence and realistic priors all the time is good (they don't do enough evidence gathering though) but it is not rationalism, it is empiricism.

Surely a rationalist would be into logic and actively refuse to read studies, like the Austrian economists did.

Re:Trapped priors as a basic problem of rationality by renewiltord on Saturday March 13, 2021 @04:29PM (#26450182)
It's just a terminology thing. Sort of like how atheist actuaries still account for Acts of God.
Re:Trapped priors as a basic problem of rationality by spiderjerusalem on Sunday March 14, 2021 @07:15AM (#26454388)
The word 'rational' is one of the most abused in western intellectual history. Hegel considered his system 'rational' and I think it is fairly self-evident that that has nothing to do with these internet 'rationalists'.
Re:Trapped priors as a basic problem of rationality by rsj_hn on Sunday March 14, 2021 @06:56AM (#26454300)
> Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately.

That sounds like an extraordinary claim that would require extraordinary evidence. No doubt physics and engineering have made great advances, but that doesn't mean that every social science that dresses itself up in the language of rationalism will be able to make similar advances, or really any advance at all. In fact, there is a case it will lead to regression rather than advancement:

Societies are very complex things made of people who respond to changing incentives. I don't believe anyone understands them very well, nor has any field in the social sciences established a track record of making highly accurate out of sample predictions, especially for the efficacy of interventions 10, 100, 1000 generations on. Even the simple question of "what keeps a society together?" is still a puzzling one, especially now that post-modern Western societies are disintegrating before our eyes and are failing to even biologically reproduce at replacement levels.

And so we turn to values. At least with values, it is possible to gain experimental knowledge in the laboratory of history. Societies reproduce values and compete with each other as well as with a hostile environment, so long lived societies should have acquired value systems more conducive to social survival.

And I don't think we can make much stronger claims than that.

Certainly someone just imagining stuff or doing some small experiments is unlikely to come up with robust rules for organizing society that will stand the test of time better than what comes from the long lived traditions of successful societies.

Re:Trapped priors as a basic problem of rationality by indrax on Sunday March 14, 2021 @01:11PM (#26456888)
Someone living 1000 or even 100 years ago did not have the information we have.(eta: to design a society) Technology also lets us afford things that past societies simply couldn't do.

We sometime learn from evolved biology to make machines, but sometimes we can also see that evolution made a bad design. By learning from what has worked it's not too difficult to design an alternate society that will work just fine for decades or centuries.

Some societies survive by preying on others. Some are stable but prey on their own. I don't want a life in service to a society that evolved to be strong. I want a society that exists for the benefit of the people in it, and improves as we learn better ways to do things. (The current system does change but different structures could be much more adaptable.)

Re:Trapped priors as a basic problem of rationality by renewiltord on Sunday March 14, 2021 @03:16PM (#26457976)
I should clarify. I meant "more accurately than not engaging in that pattern of thought" not "accurately". The latter is an absolute statement, the former a relative one.

And, yes, one can use alternative schools of epistemology if one believes so. This group of people derive theirs significantly from Popper's line of "critical rationalism" which is different from "empiricism" (the names here are the ones that annoyed our cousin comment over there).

This school has certain affordances that permit them to reason as they do but if you disagree the differences between us are likely so large that it is unlikely that I can explain without expending far more effort than I am willing to.

Re:Trapped priors as a basic problem of rationality by drdeca on Saturday March 13, 2021 @03:09PM (#26449548)
Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values.

So, for example, in the climate change example he gives, while it is of course appropriate to take into account possible biases of people promoting one position or another, and ways that those may result from the different values that the people with those biases, one should nonetheless endeavor to have a tendency to believe what is actually true about the factual matter, even if it were that the people with the wrong values were biased as an (indirect) result of those values, in a way that led them to be more likely to reach the correct conclusion about the empirical facts, than those with the good values.

(Not to say that I think this is the case in this example. Just, hypothetically. Or, what do you call something that is like a hypothetical but without expressing any position about whether the thing is actually true?)

In addition, while they may get tangled up and which is which may change for a given person over time (due to human thought being fuzzy and wobbly), I think it still makes sense to make a distinction between terminal values and instrumental values. Further, if one values something instrumentally, because one believes that it is instrumentally beneficial for one's terminal values, and it is demonstrably true that it actually does not benefit one's terminal values, then one should come to believe this truth, and so no longer instrumentally value the thing.

I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model.

People will sometimes understand their past behavior as arising from a cognitive error, and endeavor to address this. I suppose you could regard this as simply conflicting preferences, or preferences about preferences, but I don't think this is the best way to understand things.

Re:Trapped priors as a basic problem of rationality by philipkglass on Saturday March 13, 2021 @04:32PM (#26450196)
Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values.

That's true, and I have been convinced of some of those things. For example, I am empirically convinced that mass shootings by spree killers are shocking but not a major issue of public health or safety in the US. Even though I believe that Democratic politics gives disproportionate attention to mass shootings, weirdly mirroring how Republican politics gives disproportionate attention to Islamist terrorism, I'm not going to say so in the company of many Democrats. Because another thing I have been empirically convinced of is that people build effective political coalitions by emphasizing common ground and judiciously overlooking their allies' faults. I want action on climate issues, so I'm not going to alienate my natural allies by lecturing them about their irrational fear of semi-automatic rifles. Effective politics is almost the thematic opposite of a personal quest to aggressively probe all things in search of Truth.

I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model.

It's a model that doesn't lend itself to mathematically grounded predictions. But is it actually not reasonable? Without access to another person's qualia it's really easy to misattribute whether their more surprising decisions are revealing a preference, demonstrating cognitive error, or servicing a fundamentally different value orientation. Like Hume's Problem of Induction I can't effectively grapple with it but I can't exactly dismiss it either.

Re:Trapped priors as a basic problem of rationality by roenxi on Saturday March 13, 2021 @07:10PM (#26451448)
> It is probably a good strategy to refuse to be talked into positions against your fundamental values by mere evidence. I'm joking but also serious.

I believe that too. There are some boxes so dangerous we need people not to open them even if they believe they should.

However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence.

We don't get sewage, skyscrapers and subways by following our instincts or because some people somehow made them values. Everyone wants good sewage systems. The best parts of society come from following evidence and logic as best we are able - including, I point out, the legal system.

Re:Trapped priors as a basic problem of rationality by philipkglass on Saturday March 13, 2021 @07:46PM (#26451671)
However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence.

I don't think that you necessarily need to sway kids while they're young. 10 years ago there were no states in the US that had legalized recreational cannabis use for adults. Today there are 17, and I expect at least another 17 over the next decade. What changed? Attitudes shifted among all age groups. It didn't seem like a process of grandparents dying out and their grandchildren entering the voting booth.

It didn't seem like an evidence based process either. I was well persuaded in the 1990s that the deleterious health effects and other negative externalities of cannabis use were no worse than those corresponding to alcohol. The historical example of Prohibition was more than enough evidence for me that fueling a black market was worse than letting adults impair themselves. I wonder why evidence didn't persuade people to change laws sooner, yet the shift happened far faster than generational replacement once it started.

If anyone reading has a convincing model for how the cannabis attitude shift happened: does it only rationalize history, or does it also predict other things likely to change in the near future? There are a lot of ways we can turn past series of events into coherent stories. There are fewer that successfully foretell what happens in the next chapter.

Re:Trapped priors as a basic problem of rationality by salawat on Sunday March 14, 2021 @12:39PM (#26456648)
The cannabis legalization happened because of two additional factors:

A) Overincarceration of non-violent offenders

B) the reveal of racist roots in the foundational war on drugs rhetoric (which led to more strict penalties for the use of crack [cheaper street drug], for instance, than straight cocaine [high-society, white-collar drug]).

At least that seems to be the lynchpin from my point of view. It was always going to fail from mmy point of view since prohibitions in the U.S. don't have stellar records.

Re:Trapped priors as a basic problem of rationality by smcameron on Saturday March 13, 2021 @10:58PM (#26452679)
> There are some boxes so dangerous we need people not to open them even if they believe they should.

Religion is several of those boxes, which have already been opened for centuries.

Faith, (by which I mean deliberately attempting to believe something to a degree of certainty which exceeds what is warranted by the available evidence) is one of those boxes that should be sealed up and ejected into space, or at least made to be deemed immoral by the majority of people, since it obviously is immoral.

Re:Trapped priors as a basic problem of rationality by mensetmanusman on Sunday March 14, 2021 @08:58AM (#26454996)
Few people carry that definition of faith though, but many, especially during the French Revolution, felt as you do, and acted on it, not by ejecting religious people into space, but by gathering them together in barns and burning them in mass executions.

https://en.m.wikipedia.org/wiki/Infernal_columns

Re:Trapped priors as a basic problem of rationality by benchaney on Saturday March 13, 2021 @11:55PM (#26452903)
The cognitive error being discussed here relates to a systematic failures to make accurate descriptive statements. The distinction between deontological and consequentialist thinking is entirely prescriptive. Being able to make accurate descriptive statements and understanding the accuracy of descriptive statements are valuable tools regardless of if you are a consequentialist, a deontologist or adhere to some other moral framework altogether.

I believe that torture is wrong and I also believe that torture doesn't work. I could theoretically be convinced that torture works, but I would still believe that it is wrong. If I were to insist that torture doesn't work in the face of strong evidence to the contrary because of my moral beliefs that would be a cognitive error.

Regarding your footnote: They cannot be the same thing because preferences are prescriptive and errors are (at least as they are being discussed here) are descriptive.

Re:Trapped priors as a basic problem of rationality by ineptech on Saturday March 13, 2021 @11:23PM (#26452793)
> "cognitive error" and "revealed preference" may also be different names for the same thing. Are they experimentally distinguishable?

"I would still be against torture no matter what the research shows" is revealed preference, and "I'm against it, therefore it doesn't work, no matter what the research shows" is cognitive error.

This is clearer if you suppose that I published a good, thorough study showing that torture works very well. In the former case, you could accept it as accurate but retain your opinion; in the latter case you would be forced to either change your stance or accuse me of [ineptitude, fraud, bias, secret sadism, etc].

Re:Trapped priors as a basic problem of rationality by db579 on Sunday March 14, 2021 @08:38AM (#26454823)
I think this is often confused because "works" is poorly defined leading both sides to interpret it as: works == achieves my aims == matches my values
Re:Trapped priors as a basic problem of rationality by catlifeonmars on Saturday March 13, 2021 @02:50PM (#26449379)
I believe that the author makes the basic assumption that rationality is the end.
Re:Trapped priors as a basic problem of rationality by bordercases on Saturday March 13, 2021 @09:56PM (#26452394)
> But in fact many political zealots never accept reality. It's not just that they're inherently skeptical of what the other party says. It's that even when something is proven beyond a shadow of a doubt, they still won't believe it.

“We’re an empire now, and when we act, we create our own reality. And while you’re studying that reality — judiciously, as you will — we’ll act again, creating other new realities, which you can study too, and that’s how things will sort out. We’re history’s actors . . . and you, all of you, will be left to just study what we do.”

- Karl Rove

[dead]
Re:Trapped priors as a basic problem of rationality by smolder on Saturday March 13, 2021 @04:24PM (#26450136)
Your post claims to describe a good example of political zealotry and then goes on to be one. To frame Floyd's killing by police as some routine accident is ignorant of the evidence made available to you, probably because you discounted such evidence as untrustworthy and biased. Guesses about how many die to police per year are irrelevant to whether the force used was excessive.
Re:Trapped priors as a basic problem of rationality by ggreer on Saturday March 13, 2021 @04:42PM (#26450266)
I think his argument is that it’s a freak accident, not a routine one. If it were routine, the stats for unarmed people dying during arrest would be much higher.
Re:Trapped priors as a basic problem of rationality by smolder on Saturday March 13, 2021 @05:29PM (#26450707)
My phrasing was off. His argument was that the accident was nothing but a freak occurrence, and that the manner in which Floyd was restrained was routine and appropriate, which, as performed, it was not.
Re:Trapped priors as a basic problem of rationality by ilaksh on Saturday March 13, 2021 @11:32PM (#26452829)
This discussion of Floyd is a perfect example of what the article talks about.

Two radically different interpretations of the same event.

Re:Trapped priors as a basic problem of rationality by elefanten on Saturday March 13, 2021 @05:32PM (#26450736)
GP actually laid out a full argument while you dismissed it out of hand as zealotry. It really feels like you're proving GPs point.
Re:Trapped priors as a basic problem of rationality by smolder on Saturday March 13, 2021 @07:43PM (#26451649)
I skipped what are in my experience fruitless lines of argument. I cannot decisively argue about to what extent the drugs in his system may have contributed to his death, or to what extent the defunding of Minneapolis police was "dangerous", or to what extent criminality shifts blame to the criminal in these interactions when they go wrong. Nor can he, despite his beliefs.

I also skipped arguing with what I think is correct, like that media attention distorts peoples view of things.

Re:Trapped priors as a basic problem of rationality by veridies on Saturday March 13, 2021 @07:43PM (#26451650)
The post in question is full of strongly stated points, presented with minimal evidence, and in many cases irrelevant to the real argument being had. I would suspect that most people didn't respond because they recognized it as a waste of time. Not every argument is worthy of engagement.

For example:

> Floyd had high levels of drugs in his system, COVID-19, and was complaining about breathing in the car before there was a knee on him.

That makes it much more important not to use a chokehold.

> Chokeholds are also a very standard and typically non lethal form of getting a suspect under control.

This is relevant to whether Floyd's killing was an aberration; the protests were predicated on the assumption that it was not, and that police in general use unnecessary force.

> There’s no evidence of race playing a part in this incident (no evidence of motive)

As per https://www.washingtonpost.com/graphics/investigations/polic..., a higher proportion of unarmed black men are shot by police than unarmed white men. So whether or not THIS CASE was race-related, there is a racial aspect to the problem.

> or it even being an instance of police brutality (standard and typically safe procedures were used)

"Typically safe" is arguable; Floyd was not the first man killed by a chokehold in completely unnecessary circumstances, where there was no imminent need to subdue the person being arrested (see Eric Garner). But again, this is the problem. Pointing out that it's common looks like an argument against it, but it's really not.

> because of these complicating factors. The news media focus, social media virality, sustained legal protests, and sustained violent rioting have corrupted people’s ability to judge this situation rationally, to the point that people overestimate the number of unarmed black people killed by police by over 50x (https://www.lawenforcementtoday.com/poll-44-of-liberals-say-...)

This number is based on a database that documented cases where we know that police have killed someone, we know that person was black, and the police have said the person was unarmed. The police's judgement on what constitutes "unarmed" may not be the same as that of the people polled.