News for hackers. Stuff that matters.
Re:Defecting by Accident - A Flaw Common to Analytical People by ZeroGravitas on Wednesday December 01, 2010 @07:19AM (#1957638)
Bad explanations of the prisoners dilemma are a constant fascination to me. They pop-up in so many otherwise excellent bits of writing, imho, because the standard mental image of two prisoners being interrogated is such a mismatch for the actual dilemma.

For example, in this case he's talking about "defecting by accident", yet two paragraphs before he's just outlined the fact that "the best solution individually is to defect".

So you're "accidentally" following the optimal strategy, which is the same one you'd pick if you had perfect information about the situation. I fail to see how that's a problem.

The key to the prisoner's dilemma is to avoid being in that situation. Once you're in that situation, a situation that is fascinating precisely because you are so totally screwed once in it, the "game" is over. You (and your partner) only have one rational choice and it leads to a poor outcome.

Change the game when and if you can, that's the lesson of the prisoners dilemma (also, if you can't change the game then defect because the other guy should logically defect too leaving you high and dry if you try to cooperate and if for whatever irrational reason he doesn't defect then you get the best possible outcome by defecting when he doesn't).

To be extra clear, "defecting" will always gain you something in the standard Prisoner's Dilemma. That's kind of the whole point. In an iterated prisoners dilemma, where the guy you betrayed previously gets a chance to get even it's not so simple and gets more confusing if people hold irrational grudges. Interaction with colleagues obviously fits that kind of iterated version better as does pissing off your partner in crime who'll have you shanked in the prison showers as revenge for betraying him.

For a discussion about nerds needing to communicate better, leading off with a false technical analogy seems like a perfect way to piss the techies off and make them ignore the rest of your message, which all seems reasonable enough in itself and doesn't really need the reference to the PD and talk of defecting to jazz it up.

Re:Defecting by Accident - A Flaw Common to Analytical People by freeflygeek on Wednesday December 01, 2010 @08:41AM (#1957827)
I love this comment, because it is such a brilliant demonstration of the blog author's points -- let's focus on what I can do to show that I am smarter than you!
Re:Defecting by Accident - A Flaw Common to Analytical People by iansimon on Wednesday December 01, 2010 @01:29PM (#1958892)
To be fair (but really to sound smart (possibly unsuccessfully)), your comment is also an example of this.
Re:Defecting by Accident - A Flaw Common to Analytical People by mquander on Wednesday December 01, 2010 @09:41AM (#1958001)
FWIW, I think you might be misreading the audience of the post. The majority of readers at LW subscribe to not-directly-causal decision theories under which they would cooperate with another rational agent in the one-shot PD; it may distract you, but not most.

I agree with you, however, that it isn't the greatest analogy.

Re:Defecting by Accident - A Flaw Common to Analytical People by ZeroGravitas on Wednesday December 01, 2010 @09:59AM (#1958056)
Are you talking about Superrationality (http://en.wikipedia.org/wiki/Superrationality)?

Still strikes me as changing the rules and then pretending you've said something new about the original unchanged form. Kind of like saying you'd choose white in the "You must choose black game". I'm a bit disappointed if this kind of sloppy logic/mysticism is popular on LessWrong.

Re:Defecting by Accident - A Flaw Common to Analytical People by ZeroGravitas on Wednesday December 01, 2010 @10:37AM (#1958176)
Googling for mentions of this on LessWrong it seems Elizier agrees that the standard formulation is confusing but remains strangely coy about what a rational actor should/would do in response:

http://lesswrong.com/lw/tn/the_true_prisoners_dilemma/

Actually, in a follow-up he's clearer in a comment:

"As someone who rejects defection as the inevitable rational solution to both the one-shot PD and the iterated PD, I'm interested in the inconsistency of those who accept defection as the rational equilibrium in the one-shot PD, but find excuses to reject it in the finitely iterated known-horizon PD."

from http://lesswrong.com/lw/to/the_truly_iterated_prisoners_dile...

Haven't found an actual explanation of why he thinks co-operation is rational in a one-shot PD yet though. He's demonstrated that he understands the problem enough to describe why it's counter-intuitive, so I'm going to assume that whatever his reason is it's something a bit crazy, rather than just dumb.

Re:Defecting by Accident - A Flaw Common to Analytical People by pjscott on Wednesday December 01, 2010 @02:02PM (#1959057)
If two "rational" actors go into a one-shot prisoner's dilemma and both defect, then they're worse off than "irrational" actors who both cooperate. What the hell kind of rationality is that? If you're going up against an agent whom you think will defect, then sure, it's rational for you to defect as well -- but defecting is not always the right answer.

I'm guessing that Eliezer's being relatively quiet about this because he's still working on a grand unified theory of how to approach such problems:

http://wiki.lesswrong.com/wiki/Timeless_decision_theory

Re:Defecting by Accident - A Flaw Common to Analytical People by ZeroGravitas on Thursday December 02, 2010 @05:24AM (#1961681)
It's rational as defined by economists i.e. nothing to do with rationality at all. Similarly, don't be fooled into believing that perfect competition is perfect or free markets are free. It's just jargon, shorthand for a bunch of assumptions about a model, almost none of which actually hold in real life but just might, if you squint the right way and bear all the caveats in mind, have some useful information to impart.

from http://en.wikipedia.org/wiki/Rational_choice_theory

"The "rationality" described by rational choice theory is different from the colloquial and most philosophical uses of the word. For most people, "rationality" means "sane," "in a thoughtful clear-headed manner," or knowing and doing what's healthy in the long term. Rational choice theory uses a specific and narrower definition of "rationality" simply to mean that an individual acts as if balancing costs against benefits to arrive at action that maximizes personal advantage.[4] For example, this may involve kissing someone, cheating on a test, using cocaine, or murdering someone. In rational choice theory, all decisions, crazy or sane, are postulated as mimicking such a "rational" process."

Also, note that co-operating with the other person gives the greatest total gain (i.e. adding yours and the partners) but the greatest individual gain is when one defects and the other tries to co-operate. So betraying the other is the best possible outcome, and he knows that, and knows you know that. That's why you can't trust them to co-operate, they have no incentive to do so.

Re:Defecting by Accident - A Flaw Common to Analytical People by mquander on Wednesday December 01, 2010 @10:28AM (#1958152)
Yes, I'm talking about superrationality and decision theories that accommodate it, like evidential decision theory (you should make decisions that serve as evidence for the outcome you desire; e.g. you should cooperate, because doing so is evidence that a corresponding EDT-using rational partner will also cooperate) and Eliezer's pet timeless decision theory (roughly, your decisions should be treated as if you were making the decision for all sufficiently similar minds in the same situation.)
Re:Defecting by Accident - A Flaw Common to Analytical People by ZeroGravitas on Wednesday December 01, 2010 @11:25AM (#1958364)
Thanks for the pointers, I found some stuff on their wiki with these keywords and it looks like some interesting reading.

However, from a quick look through this interpretation makes it an even worse analogy, even if you overlook not flagging it up as an unconventional take on PD for outsiders.

Re:Defecting by Accident - A Flaw Common to Analytical People by mquander on Wednesday December 01, 2010 @11:37AM (#1958428)
Yeah, I agree! All I took away from the analogy was "you had in mind one thing that you wanted to do, accidentally did the other thing, and suffered for it."