Yes, I'm not being clear. To elucidate, I'm not saying they chose the correct responses, I'm saying most respondents chose the answers that were quantitatively closest to the correct responses, without actually choosing the correct responses.

So if:

a = 5

b = 10

c = 15

d = 20

And the correct answer is d, I'm saying most respondents chose c, for most of the questions. We could calculate the odds of them doing this for 9 out of 10 questions if they were choosing at random: the binomial distribution shows us that it's a very small probability (I calculate 0.0000286102294921875, but that may be off a little).

It's a little more complicated here we're dealing with the average probabilities of many individual events, and to be honest I'm not sure how to account for that (if it even makes any difference, perhaps a statistician can chip in?). But they're almost certainly doing much better than chimpanzees.

This is incorrect for all but one (maybe two) of the questions. The one you specifically mentioned, about literacy, is totally wrong. The answers are just about evenly distributed between 40% and 60%, with far more choosing 20% over the correct answer: 80%.

I think jnbiche has a potentially interesting point. There are only five sample questions revealed in the article, however, so we can't know, unless we have access to all the data, whether his claimed pattern actually holds.

Of the five, four are quantitative. Of those, #1 does not follow jnbiche's pattern; #2, the literacy question, does follow the pattern if you look at the "all respondents" number, but, as you suggest, the numbers are too close; #3 follows the pattern, but there are two answers equidistant from the correct answer, which complicates the analysis; #5 does follow the pattern.

Rosling's discovery, that we do worse than random on tests like these, is fascinating, despite so many people here becoming distracted by the rhetorical device of the chimpanzee. But that only holds if you merely tabulate right and wrong answers. jnbiche is pointing out that there might be a deeper pattern to the data that could make us look better, perhaps almost as good as random.

I wonder how easy it would be to design other surveys that exploited bad preconceptions to get a worse-than-random result. Maybe in other fields, like science.