News for hackers. Stuff that matters.
Re:Scotland rejects independence by throwaway283719 on Friday September 19, 2014 @04:36AM (#8339485)
I think it's interesting to talk about the failure of polls to predict the magnitude of the result. Here are the results of the final polls from six major polling organizations, taken 1-3 days before the vote

  Organization    Yes     No      Number*
  ---------------------------------------
  Opinium         0.479   0.521   1087
  ICM             0.477   0.523   1010
  Panelbase       0.474   0.526    954
  Survation       0.473   0.527    910
  YouGov          0.474   0.526   3043
  Ipsos Mori      0.474   0.526    951
  ---------------------------------------
  TOTAL           0.475   0.525   7955

  * excluding undecided voters
On one hand they were accurate in that they forecast a "no" result with high probability - the proportion of "no" voters is more than four standard errors away from 50%; the standard error being

  s.e. = sqrt(0.475 * 0.525 / 7955) = 0.006
On the other hand, the final proportion of "no" voters was 55.3%, which is more than four standard errors away from the forecast value of 52.5%.

So the polls were significantly underestimating the proportion of "no" voters out there. A similar effect was seen in the Quebec independence referendum, where the polls actually predicted a slight advantage for "yes" but the final result was a decisive victory for "no".

Re:Scotland rejects independence by oliwarner on Friday September 19, 2014 @04:48AM (#8339531)
And there were notes that went along with these figures. Certainly YouGov's. They said that if correct, their figures were likely going to be multiple-points out in favour of No because of various social factors:

- "No" is fundamentally negative, people don't like to be negative - Supporting the Conservative party is seen as violently offensive anywhere north Sheffield. People that do are less likely to give their opinion in public.

Given the lack of precedence, these are factors they couldn't adjust for.

Re:Scotland rejects independence by UVB-76 on Friday September 19, 2014 @04:44AM (#8339519)
Independence was the high-risk option.

I don't think it's surprising that people are more risk-averse on polling day, when their vote actually matters.

Re:Scotland rejects independence by adaml_623 on Friday September 19, 2014 @04:42AM (#8339509)
I'm not sure because I don't know where you've sourced your data but where are the figures for undecided voters? That was always a large number and explains why your s.e. is too small.
Re:Scotland rejects independence by throwaway283719 on Friday September 19, 2014 @04:50AM (#8339541)
That's a good point, and I've now corrected for that, but it makes very little difference to the top line result (that the polls underestimated the support for "no" by a large margin).

The proportion of undecided voters in these polls was about 7.1%.

Source is http://whatscotlandthinks.org/opinion-polls

Re:Scotland rejects independence by jiggy2011 on Friday September 19, 2014 @04:42AM (#8339510)
My guess is that a lot of people like the idea of an independent Scotland but are frightened of the reality and bottled it on the way to the polling station.
Re:Scotland rejects independence by philjohn on Friday September 19, 2014 @10:15AM (#8340581)
Re:Scotland rejects independence by mike_hearn on Friday September 19, 2014 @04:37AM (#8339491)
Re:Scotland rejects independence by nknighthb on Friday September 19, 2014 @04:51AM (#8339547)
A related question I'd be curious about is the likely voter issue. In the US, left-wingers are less likely to actually go vote than right-wingers. Pollsters try to correct for this with a "likely voter" model (which varies by pollster, but tries to discover how likely a respondent actually is to vote).

If you poll everyone eligible to vote, you get left-wing results. If you poll registered voters, it pulls a bit right. If you poll voters that match a reasonable "likely voter" model, you'll get results that are even further right and tend to be closer to the reality of votes actually cast.

Does this dichotomy exist in the UK and/or Scotland specifically, and if so, do pollsters there try to correct for it?

Re:Scotland rejects independence by adaml_623 on Friday September 19, 2014 @05:46AM (#8339683)
I read that a problem the pollsters were having was simply that they didn't have any precedents. There was a referendum last century asking a different question but that wasn't that helpful. Essentially you can't make up a model without observations and opinion polls don't count
Re:Scotland rejects independence by soneil on Friday September 19, 2014 @08:33AM (#8340167)
This is my understanding of the issue too. Polls don't just ask the question. Each cycle they take their input, the actual results, and refine their model. Over iterations the model should get more and more accurate.

This event is a one-off. No cycles, no iterations, leaving a model which is essentially a best-guess.

Re:Scotland rejects independence by M2Ys4U on Friday September 19, 2014 @10:32PM (#8343495)
Some pollsters ask how likely one is to turn out to vote.

Amongst those that do, they weight responses differently.

Some will only report those who are 10/10, 9/10 and 8/10 likely to vote in their topline figures.

Some will multiply the likeliness with the voting intention, so that a 10/10 likely voter would count as a whole vote, a 9/10 likely voter as 9/10ths of a vote and so on.

The UK Polling Report[0] - run by the guy in charge of YouGov, if I recall correctly - is a very useful site for explaining UK polling oddities!

[0] http://ukpollingreport.co.uk/