They were generally correct with their prediction except in 3 states where nobody had been doing detailed polling because the pollsters didn't think it would matter.

It simply wouldn't have been possible for the models to be more accurate with the data they had. As they say, bad data in, bad data out.

That's why this time around the pollsters made sure to be more thorough in their polling.

> That's why this time around the pollsters made sure to be more thorough in their polling.

"This time is different."

I've heard that enough times to be highly skeptical. I'm also deeply skeptical of the notion that polling is even remotely correlated to actual results. Cultural and historical trends play a drastically higher role and are almost always left out.

> I'm also deeply skeptical of the notion that polling is even remotely correlated to actual results.

But it has been strongly correlated to the results in basically all elections so far in all democracies on the planet. Taking 2016 as an example there has been a very strong correlation between polling and the results. The national polling averages were only 3 points off from the actual result. If that's not correlated I don't know what is.

Polarization and the unacceptability of publicly saying "I voted for X" also didn't really exist prior to 2016. The fear of getting doxxed, combined with a record low level of trust in institutions and the media, leads to skepticism toward answering polls truthfully, IMO.

While polarization is bad now, it's nowhere close to historical extremes, and it's not even as bad as it was during the post-Vietnam era only a few decades ago.

As for fear of doxing: plenty of people openly supported and voted for George Wallace (a noted white Supremacist) back in the day, and even Roy Moore (accused pedophile) just 2 years ago. Proud Boys members openly pose for the cameras even as they espouse racist views, and QAnon members brag about being part of QAnon.

The purported shy Trump voter? Doesn't exist. https://fivethirtyeight.com/features/trump-supporters-arent-...

As a data point, I live in a conservative-leaning area of a purple/blue state. Along my normal driving routes I had seen quite a few, but over the past few weeks they have mostly been taken down (in every case, other republican candidate signs still stand). Scenario one, Trump supporters supported him all the way up until now, enough to donate to his re-election campaign to buy a yard sign, and in the final weeks of the election, decided that they'd had enough and taken down their signs. Scenario two, some random stranger decided that it's their right, nay, their responsibility, to moderate someone else's yard signs.

Not every Trump voter is a "Proud Boy" or noted white supremacist, maybe even some of your friends are who "just aren't interested in politics" because they know that if they were honest with you, you would flip out. Some understand that this attitude about yard signs is actually representative of an entire worldview, and opposing that might actually be the lesser of two evils.

Similar to how "not every Trump Voter is a >>Proud Boy<<" not every Biden supporter is a yard-sign stealing communist. Actually the extremists are in the minority in both groups. If you start going down that road and base your vote on how bad the worst people on the other side are, democracy is pretty much collapsing already.

Sure - I am above my quota for politics posts on HN but since this article is about problematic tail behavior I'll go ahead anyways. You are right that probably a very small minority of Trump opposers are actually stealing yard signs. However, a larger component are advocating for censorship of opinions and facts that disagree with their worldview, and many more will readily shut down any opposing discussion about immigration policy or international trade, supporting police, or even simply wearing a red hat, with strong and viral accusations of "racism" - signalling to their allies that this person should be punished, and many of those allies are happy to comply in whatever way their position in society allows them to. These attitudes and behaviors are pervasive and malignant; the yard signs are only a visible symptom of a larger problem.

I just remembered this thread when I came across this story of a club owner being harassed after appearing in an Ad for Joe Biden https://www.billboard.com/amp/articles/news/politics/9471700....

Essentially I think that those narratives about one side "shutting down" the other side are not true for either party. In my opinion they are mainly the result of filter bubbles where you are constantly presented to all the outrageous things that the other side is doing (the above mentioned article is clearly an example) while actually people remain largely civil and are not tied to their political identity in such an extreme way.

The shy Tory factor[0] is likely to be even stronger this year than in 2016 when it comes to polls. After 4+ years of being harangued and called every name in the book by the vast majority of national culture (movies, music, TV, news, social media, news-entertainment), along with increasingly hostile projects such as https://donaldtrump.watch/ I imagine less of his more subdued supporters are going to be honest with pollsters.

[0]: https://en.wikipedia.org/wiki/Shy_Tory_factor

But wouldn't that mostly be concentrated in places/areas where their votes aren't likely to matter? Having just driven through the US South in the last month, I can confidently tell you people are not in any way shy about their support for Trump. I saw more Trump signs and flags than I saw US flags.

It's also likely this works in both directions - if you support Biden, I bet you don't have a yard sign for it if you live in Mississippi.

By construction, the effect is strongest the more you're not in the majority, which also means your unspoken support is more likely to not matter on the actual outcomes of the election.

It's more relevant in swing states, which are by definition mixed. I.e. if you live in Pennsylvania, Michigan, Ohio, or Florida, you can't really be sure how your neighbors will react to a Biden/Trump sign.

If you're right, the support for Trump should be higher in polls where people talk through a phone menu or the internet, compared to live interviews.

Newsflash, they aren't, so your hypothesis must be they're too shy to admit their Trump support to a machine?

Everybody is just fighting the last war where Trump suddenly won for a couple of reasons. So, the Dems are scared they are missing something, and the Reps are going "Haha, who cares about modelling".

Will, the forecasts be perfect? Nope. But is the margin rather large, but not unsurmountable? Yes it is. Are the mistakes from last time repaired? Yes, they take care of uneducated whites. Is there evidence Trump has found a new source of voter support? I haven't seen it.

Yep. I see this everywhere, both online and in person.

Anecdotal, but I have never met a Trump supporter that was shy about who they were voting for, this year or in 2016.

If anything, Trump supporters have been extremely vocal about who they were supporting, to the extent that they frequently violate social norms and try to takeover events and gatherings to make their political affiliations known, like this week with Among Us.

Saying that the ultimate outcome had a 1 in 4 chance is not wrong, slightly wrong, or less wrong. If the weatherman says there's a 1 in 4 chance of rain, and it rains, he wasn't wrong.

To take it a step further, if that was the forecast 4 days in a row, you would expect it to rain one of those days.

Not necessarily.

There is a 0.750.750.75*0.75 or a 31% chance of no rain at all

I'm not saying it's certain, but it's what I'd expect. It raining on one day out of 4 is the most common case.

No, but it means that the weatherman isn't particularly effective at forecasting the weather.

No it doesn’t, and this is a fundamental misunderstanding of how probabilistic forecasting works. If it rains 9 out of 10 times a weatherman says there is a 30% chance of rain, they are a bad weatherman, but they aren’t much worse than if it rained 0 out of 10 times they predicted a 30% chance of rain. A weatherman accurately assessing the probability of the weather forecast would see it rain around 3 out of 10 days they say there is a 30% chance of rain.

Only if they say that every day, and it rains every day. If the most likely outcome happened every time, then the model is likely wrong/under-confident. The prediction is never going to be 100% accurate until the event is happening/has happened. Up until then, there's always a chance you're wrong or something can change. Being wrong once isn't necessarily a sign that the whole system is messed up.

This is a fundamental misunderstanding of probability. Low probability events do happen, and it doesn't inherently mean the estimated probability was wrong.

I am criticizing their labeling of Trump's win as low probability.

"He won" is not sufficient evidence upon which to do that.

Nowhere did I say it is. I simply think that, if one were following the right information, his win was not as unexpected as the coastal media presented it as being.

Personally I would have put it about 60-40 Hillary-Trump.

You started off by calling the 2016 prediction a debacle, but now you're saying you would have put the odds about 12 percentage points differently. That doesn't seem like a big enough disagreement to warrant the kind of vehement criticism you're throwing around.

I called it a debacle because 99.9% of media sources, pundits, politicians, political figures, or anyone else thought that Hillary had anything less than a guaranteed win. I'm simply suggesting it was actually always a close race, but that the media ignored this because it went against their ideological model / they weren't familiar with places like the Rust Belt.

You really need to be more clear about which of your criticisms are against FiveThirtyEight specifically vs against the media in general. Because now it's looking like you are complaining about the media in general and using that as justification for mistrusting FiveThirtyEight, in the context of a discussion specifically about FiveThirtyEight being a notable outlier from that general media trend.

It's still just unclear to me how FiveThirtyEight assigning Trump a ±30% chance of winning can be considered "accurate" or "good".

From my point of view, this is only because said people considered Trump winning so extremely unlikely that 538 getting it sort of right appears exceptional. In reality, they were still quite wrong, just slightly less so. Ergo I don't see much value in their model.

I'm seeing the same exact thing today with Biden at a 90% chance of winning.

At the end of the 2016 election, Donald Trump had a predicted 1/4 chance of winning the presidency. Does this seem like a massive debacle that Trump won under these conditions? Not to me.

https://projects.fivethirtyeight.com/2016-election-forecast/

They said Trump had a 1 in 4 chance. That's very high. NYT had something like 1 in 20 chance for Trump.

That doesn't really answer my question. It only indicates that they were slightly less wrong than every other media source, not that they have a good model.

If I had a laptop that only worked 1/4th of the time, rather than 1/20th of the time, would that make it a reliable laptop? I don't think so.

If they were wrong, but “less wrong” than all others, you should pick their model (unless you have an oracle, because the alternative - flipping a coin or “relying on your intuition” is rarely better).

Also, it doesnt make sense to look at a single prediction to evaluate a model.

Out of all the predictions they have made (did you look at individual state predictions?), how many were correct (and how confident were they?) - how many were wrong (and how close to 50% were they?).

That is how you evaluate a model (aka cross entropy)

It's not "wrong" to predict a low chance for something that eventually happens. Unlikely events can happen.

That isn't the criticism. The criticism is the appellation of it being "unlikely."

For example: anyone paying attention to the Rust Belt ±1980-2016 would have dramatically upped Trump's chances in Pennsylvania and Michigan. FiveThirtyEight had Hillary with 70%+ chance of winning both, which to me, shows a deep ignorance of actual cultural factors.

> anyone paying attention to the Rust Belt ±1980-2016 would have dramatically upped Trump's chances in Pennsylvania and Michigan.

There was a very decent chance that Clinton could have won in 2016 (if any factor had gone slightly better for her), and if that had happened, nobody would be saying this now. This is literal hindsight bias.

Aren't you doing the exact same thing that you're accusing me of?

My view is simple: the media completely, totally got 2016 wrong, mostly for sociological reasons. The people making the predictions simply had a huge blind spot. Brexit is another similar situation. The fact that Hillary almost won or Brexit almost didn't happen isn't really the point, because both things were never expected to be even remotely that close. Had the predictions been "Pennsylvania will be close", it would be relevant, but those weren't the predictions.

It seems like you are conflating probabilities with absolute certainties. If I had a 1:10 probability of winning the lottery, I would probably take it. If I had a 1:20 probability of getting injured if I leave my house today, I’d stay home. If I did get out but didn’t get injured doesn’t mean the model was wrong.

If you have a die that rolls a one 1/6 of the time, do you consider the die wrong?

No, but I don't consider it useful toward predicting the outcome, which is of course what this is all about.

Say more about "but I don't consider it useful toward predicting the outcome"

How else would one predict the outcome of a die roll, specifically?

If there was sufficient data to assign a 0% or 100% probability to an event, that’s what a forecaster should do. If there isn’t sufficient data, then anyone who claims there is a sure thing is a charlatan.

If I tell you you're not likely to get two heads in a row, and you do, does that make me un-reliable?

It's unfortunate we can't just run the election again a few times, and actually find the rate at which Trump is elected given the polls.

And it's not empty signalling if 538 assigned Trump a higher chance of winning; they were pretty much the only ones saying he has a chance. That is why people think the models are useful.

If everyone was wrong, it is reasonable to believe a low probability event occurred. However, given the extent to which people predicted a Trump loss (say 1 in 20), which is significantly rarer, given that the event occurred it suggests the model that predicted a Trump win with the greatest probability to likely be a more accurate model.