The NYT I agree with, but Five Thirty Eight specifically called out Trump's chances in the final days before the election: https://fivethirtyeight.com/features/trump-is-just-a-normal-...

538 analyzed the various polls--and corrected for their methodologies and predictive power--just fine!

A sibling comment brings up a batter, and I think that's a great analogy. A good batter has a batting average of, say 0.273. Nobody is shocked when they hit the ball!

538 gave a similar chance to Trump. He hit the ball.

Yes and no. 538 published an article after the election explaining that there were in fact systemic errors in the models; the result wasn't just random probability within models that were otherwise correct: https://fivethirtyeight.com/features/the-polls-missed-trump-...

> While the errors were nationwide, they were spread unevenly. The more whites without college degrees were in a state, the more Trump outperformed his FiveThirtyEight polls-only adjusted polling average,1 suggesting the polls underestimated his support with that group. And the bigger the lead we forecast for Trump, the more he outperformed his polls.2 In the average state won by Trump, the polls missed by an average of 7.4 percentage points (in either direction); in Clinton states, they missed by an average of 3.7 points. It’s typical for polls to miss in states that aren’t close, though. The most important concentration of polling errors was regional: Polls understated Trump’s margin by 4 points or more in a group of Midwestern states that he was expected to mostly lose but mostly won: Iowa, Ohio, Pennsylvania, Michigan, Wisconsin and Minnesota.

If the errors had been random rather than systematic, it would have been statistically quite unlikely for Trump to have swept all of those midwestern states.

AFAICT the polls had a systemic error AND the 538 models included the possibility of a systemic error. It's really the only way that they could have given Trump a 28% chance in the first place.

Of course, the polls themselves also acknowledge the chance of a systemic error. They're reported similar to +/- 2% 19 times out of 20. They make no claim that the 19/20 of multiple polls are not correlated.

> AFAICT the polls had a systemic error AND the 538 models included the possibility of a systemic error.

Well, 538 model took into account historical evidence that election differences from polling averages between states are strongly correlated rather than independent; arguably, that's an effect that exists in part because such deviations are in part due to systemic errors (though the particular systemic errors may differ from cycle to cycle) in the polling, rather than pure sampling error.

The +/- in a given poll is measurement error derived from the sample size. They don’t account for systemic error.

Right, but 538 did account for such an error in coming up with their ~75/25% outcome chances. Had polling errors just been random, the overall prediction would have been more like 95/5%.

But their key insight is that polling errors are often correlated, and not just statistical sampling errors that can randomly offset in either direction.

> The +/- in a given poll is measurement error derived from the sample size. They don’t account for systemic error.

Sure, but even the worst poll-based prediction outfit isn't just aggregating poll results and using the uncertainty based on sample size to determine the probabilities of different outcomes.

Pretty much all of them are using some model derived from the past relationships of polls to election results, which will, to a greater or lesser extent, capture systemic polling error that is not unique to the current cycle in the model.

19/20 does, though.

What is 19/20?

Polls are reported with fine print similar to "+/- 2%, 19 times out of 20".

I thought 538 was one of the few outlets to give Trump a decent chance? Granted, I was 16 but I seem to remember that being the case.

People _really_ struggle with probabilities (which is why 538's 2020 model page has a cartoon fox to remind them about probabilities, and headlines with visualizations). To a lot of people, saying "there's an 80% chance of THING happening" means "THING will definitely happen".

You remember correctly. "538 thought Clinton had a lock on it" is a meme, but it doesn't seem grounded in anything.

538 gave Clinton an 86% chance for a brief period of time after the Access Hollywood tapes but before Comey's press conference. I wouldn't call that a lock, but maybe some would.

86% is about a one in seven chance of not happening. It's very much not what anyone would treat as a lock on anything that had important consequences.

I grant you plenty of people have sufficient trouble understanding probabilities that they would mistake 86% for a lock, though.

Yeah, and then nobody bothered to watch as her chances drifted downward every subsequent day until the election... and they drifted down because the polls changed and the model got more information. It's not really fair to say the model was wrong- it just wasn't clairvoyant.

Really, Trump won by a hair's breadth. 538 shouldn't have given him a huge chance to win- he just barely scraped through. It's a genuinely hard problem when the popular vote is very close, since the electoral college throws a huge chaos wrench into everything.,

What should the number have been, then? The final results were essentially a tie with Trump winning the tie breaking coin toss. So 50/50 is right in hind sight. Te systemic error could just have been as easily in Clinton's favour as Trump's, so the number should have been smaller than 50% for Trump. Maybe not as small as 28%, but they weren't far off, IMO.

The model predicts a chance of winning, it doesn’t reflect the final vote percentages. In other words, a win is a win, regardless of whether it’s by a few votes or a landslide. That’s what the 80/20 represented. This is different than the polling averages. Though, electoral vote percentages are captured in 538’s model — they show a spread of possible win scenarios based on many numbers of runs of the model.

One could imagine such a non-linear model, but one wouldn't consider it well-behaved or even particularly useful for aiding one's understanding of reality.

The votes coming down to a dead heat doesn't mean that, the day before, they were equally likely to win. Suppose there were 100 possible futures. In 90 of them, Alice wins, by a little or a lot. In 8 of them, Bob barely wins, and in 2, Bob wins comfortably. Maybe this is determined by rigged dice or something, to make it strictly mechanical: there's 100 possibilities, and we know what they are and how they're distributed.

Just because you end up in the 9/100 future doesn't mean that the "real" probability was 50%. It's weak evidence that Bob's win scenarios mostly didn't involve blowouts.

It's like barely winning a sprint against Usain Bolt: maybe he had an injury midway through. Doesn't mean that he and I were equally likely to win.

The "score" was 1,405,280 to 1,382,536. That was the count in Wisconsin. Looks like a virtual tie to me.

That's the information we have, and it's real, so it's much more valid than any poll which poorly samples a few thousand people.

Maybe it was a fluke result. But that's unlikely because the other Midwestern states had similar results. But a fluke in which direction? You don't know, so it's even odds in both directions.

Perhaps people read 538's odds numbers as poll numbers.

You're right. They had him on about 33%, depending on exactly when you look.

Interestingly, in 2016 they thought Trump was more likely than the betting markets did. Betfair had him on about 22%. So if they were betting they'd have made a profit.

This year the situation has reversed. Betfair has 35% for Trump, whereas 538 has him on only 12%. So they'd be betting on Biden.

You know what they say, armies go to battle equipped for the last war that they fought. We all take our lessons learned, and recent ones seem to dominate.

So yeah, a lot of people seem convinced that since the pre-election polling showed Trump at a disadvantage last time, we'll see an identical surprise victory this time. I personally think that makes as much sense as leaving your umbrella at home on a day with a 75% chance of rain because the last time there was a similar forecast, no rain fell.

They gave Trump 28.6 which was beyond problematic. You can give baseball analogies but they were wrong. All that bad polling lead into people believing some really dumb things about the election. They've published several articles since then on why the polling is better now. Heck, even their article on being more bullish was tepid. The pre-election article they wrote[1] should have made them actually examine what they were doing. It did not. IDP/TIPP and USC/LA Times got it right.

I get that 538 is an HN favorite, but they are too damn politically biased for their own good.

1) https://fivethirtyeight.com/features/how-i-acted-like-a-pund...

I do not understand how 28.6% is "problematic," if that's roughly a 1/3 chance. That means, every third election, Trump would win with those odds - far from impossible, and it happened.

Yes, they screwed up the primary, then acknowledged and fixed their mistake. It's a large part of the reason they did much better than everyone else on the election.

> They gave Trump 28.6 which was beyond problematic.

I don't see where you've said anything that comes close to supporting such a strong assertion.

2016 was FiveThirtyEight's third time forecasting a presidential election. Getting one of those three "wrong" when they gave themselves slightly less than one in three chances of getting the 2016 election "wrong" doesn't seem at all surprising. It sounds like you're demanding that FiveThirtyEight make their forecasts deliberately underconfident but still expecting them to call the correct winner every time.

> They gave Trump 28.6 which was beyond problematic

How was that problematic?

I thought 538 gave Trump a 25% chance to win in 2016? Sure, he wasn't the favorite, but that's like the probability of a typical batter getting a hit in baseball. It isn't what happens in most at-bats, but it shouldn't be a big surprise either.

Correct. On Election Day 2016, they had him at 28.6%.

https://projects.fivethirtyeight.com/2016-election-forecast/