News for hackers. Stuff that matters.

A comment on How (not) to forecast an election: Analysis via a hierarchical Bayesian model.

Re:How (not) to forecast an election: Analysis via a hierarc... by imh on Sunday November 27, 2016 @01:29PM (#13050534)
538 made an interesting point about these kinds of biases. Essentially, polling errors are correlated. You can't unbias your estimate, but you can realize that if voters are reluctant to state their Trump preference in Wisconsin, then they are also likely to be reluctant in Michigan. If you know how correlated your errors are, you can properly account it in your own uncertainty, probably leading to a prediction that is closer to 50-50.
[ Parent | on HN ]
Re:How (not) to forecast an election: Analysis via a hierarc... by gwern on Sunday November 27, 2016 @02:41PM (#13051001)
Andrew Gelman notes that if you look at historical polls and check the total error vs just the sampling error, the latter is about half the size (implying that systematic error is as large as sampling error, but unfortunately much harder to deal with as it is unsystematic over time), and the 2016 election margin fell within the confidence interval you would expect from that: http://www.nytimes.com/2016/10/06/upshot/when-you-hear-the-m... http://researchdmr.com/ProbabilityTotalError.pdf http://andrewgelman.com/2016/11/11/david-rothschild-sharad-g...

And statisticians have pointed out the inherent limits to surveys/polls in election forecasting since at least the Literary Digest debacle; a particularly good discussion is in Schlaifer 1959 https://www.gwern.net/docs/statistics/1959-schlaifer-probabi... (excerpt: https://www.reddit.com/r/slatestarcodex/comments/5c10jx/post... )

[ Parent | on HN ]
Re:How (not) to forecast an election: Analysis via a hierarc... by whybroke on Sunday November 27, 2016 @04:53PM (#13051775)
Actually the discrepancy stems from underestimating voter turnout for Trump and over estimating it for Clinton and is not caused by the mythical timid Trump voter.
[ Parent | on HN ]
Re:How (not) to forecast an election: Analysis via a hierarc... by imh on Monday November 28, 2016 @11:47AM (#13056591)
Doesn't matter. All that matters is that the errors are correlated. If voter turnout is biased in one state, then it likely is in another. The reluctance was just an example.
[ Parent | on HN ]