I think Professor Velleman promotes "Voluntary response data are worthless" as a slogan for the same reason an earlier generation of statisticians taught their students the slogan "correlation does not imply causation." That's because common human cognitive errors run strongly in one direction on each issue, so the slogan has take the cognitive error head-on. Of course, a distinct pattern in voluntary responses tells us SOMETHING (maybe about what kind of people come forward to respond), just as a correlation tells us SOMETHING (maybe about a lurking variable correlated with both things we observe), but it doesn't tell us enough to warrant a firm conclusion about facts of the world. The Literary Digest poll

http://aurora.wells.edu/~srs/Math151-Fall02/Litdigest.htm

is a spectacular historical example of a voluntary response poll that didn't give a correct picture of reality at all.

It's effectively worthless because it needs to be qualified so heavily. Dropping the qualification (as will happen sometimes) makes it more like misinformation, which is worse than worthless, so it averages out to zero...

"The median poll selection of a HN visitor who clicked on a poll which was on the front page on the following weekdays (and national holidays in the following countries:...) was..." etc.

If the information content is sufficiently hard to extract usefully that it would be easier to redo the sample in a sensible way, then you could call the sample worthless. (A bit like uneconomic oil reserves).

So, doesn't that just mean that a statistically significant portion of the regular HN users would have to respond?

Of course, you'd have to figure out how to define "regular HN users" and what would be a significant portion of them.

If this poll were to get that much response, it would also tell you something interesting: that HN users respond to polls in statistically significant numbers :)

So, voluntary response polls are not worthless if enough people voluntarily respond to them, but this is a tricky problem.

> So, doesn't that just mean that a statistically significant portion of the regular HN users would have to respond?

No. If 50% of HN visitors voted but 99% of those over 40 didn't (due to whatever confounding factor you like - say embarrasment), you'd still wouldn't be able to extrapolate from the sample to the category of "HN visitors".

Correct. It doesn't matter how large a sample size is if the sample is biased. This should be something that every high school student knows after any high school statistics course, but it is a crucial consideration that is widely ignored.

although we can all think of an edge case.. (especially if we know how the sample is biased)

eg: a gender survey in the general population that transgender respondees do not answer -- ends up with a 99% sample size (all but the transgenders) -- and thus gives a pretty significant bit of data on male:female ratios, etc..

Agreed in principle though.

No information is better than misinformation.

I'd argue that entirely depends on the context :) it's a bit of a blanket statement.

True some of the time, but not all of the time.