"Anyway, yes, the fundamental problem in these questions is not how people think -- it's that the question they want the respondent to think they're asking, and the question they claim they're actually asking, are two different things.
The lawyer/engineer one is a classic example of this; what they hope is that you will read it as "how likely is it that these personality traits correlate to an engineer", so that they can then swoop in and say "what we were really asking is the mathematical definition of a percentage!"
Which ultimately tells us very little about the respondents and quite a bit about the people conducting the quiz..."
Either way, the article's explanation for that particular question is wrong.
"Dick is a 30-year-old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues.
This description was intended to convey no information relevant to the question of whether Dick is an engineer or a lawyer."
The description in the quiz is very different.
Engineering is more male-heavy than lawyering.
let's say we have 10 engineers, 9 of them are male. we also have 10 lawyers, 6 of them are male. Let's say one in 10 people likes doing math on the weekend.
There are 90 out of 100 ways to have a group of male engineers, one of which who likes math, but only 60 out of 100 ways to do the same with a male lawyer. furthermore, if we add in the four kids as another 1 in 10 thing, the situation gets even worse. This isn't bayesian, this is just counting boxes on a permutation table.
It's the same thing. Bayes' theorem allows you to shortcut straight to the answer without having to draw out a full probability tree / permutation table. But the underlying math is the same - in each case you have a different probability of B given A, versus B given (not A).
But then yes, ideally you could use Bayes' Rule to find that probability.
A better way to explain the common response to question #2 would be "quiz questions usually only give me information relavant to their answers" heuristic.
Given that the only distinguishing characteristic mentioned is a boolean (engineer or lawyer), and that they only chose one sample, the probability might just as well be 50% - the probability of the person picking the sample deciding one way or the other.
To the question "Are you an engineer?", Jack answered "Yes".
Would you still argue that the probability is 30% that he is an engineer? A lawyer can claim to be an engineer, after all. However I think it is clear that if we actually did the experiment, it would be much more likely than 30% that Jack is an engineer. A way of testing what you really believe the probability to be is this: I bet you a dollar that Jack is an engineer. If you wouldn't, that means that you really believe the probability to be larger than 50%.
This is because the probability that he answers yes to the question is much higher when he is in fact an engineer than when he is a lawyer. Bayes' law says:
P(E|Y) = P(E) * P(Y|E)/P(Y)
This same law applies to other characteristics, for example Y = "person likes mathematics".
Assuming that there is a skew of preferences, then this info isn't irrelevant you can perfectly reasonably use this to help identify the likelihood of this person being in one group or another. It doesn't guarantee that you're right, but it will improve your chances.
Getting hung up over the specificity of the hobbies and interests and the likelihood of those hobbies and interests representing either or a lawyer or an engineer is irrelevant, because the only factual data that was provided by the questioner is that 30% of the participants were engineers, and 70% were lawyers.
What this article is trying to present is heuristic errors - like question 1, where ignoring of the fact that sample size is relevant gets you to the wrong answer. Ignoring the likelihood that there is a correlation between personal interests and career choice seems to me to be the equivalent heuristic error for this question.
Let me give you an alternate example. There are roughly 700 million Europeans and roughly 300 million Americans. If I randomly picked one person out from this, gave you no other information and asked you where they came from, you'd have a 70% chance of guessing correctly by saying "Europe". If I told you that their first language was English, that they loved American football and baseball and hated soccer, and that their favourite TV show was Conan, and then asked you to guess where they came from, it would be hugely naive to ignore that information and still assume that they were probably European. Yes, it's entirely possible that there are Europeans who fall into all of those things, and I've not done a survey to find the exact percentage of each group that answer this description, but I'd be prepared to put a fair amount of money on the fact that there's a larger overall number of Americans who answer it than Europeans, so the smart guess would now be that they are American.
The more I see this trend, the more stubbornly I find myself clinging to "What I know"
I would bet good money that hes more likely an engineer than a lawyer.
(spoilers)
is only correct in a world where lawyers and engineers have all the same characteristics. That the sampled individual is a man will skew things all by itself.
Of the lawyers, approximately 40% will be women, whereas only 11% of the engineers. So our samplee could be one of 27 engineers or 42 lawyers - we've already bumped Peng from .3 to .39! That he likes math puzzles easily takes Peng to over .4, meaning the answer can't be A).