I guess between this and the other commenter who made money betting 538's model on predictit, I consider this pretty thoroughly debunked.
I saw some sports betting sites were offering odds, but not really a true market like predictit I think
In order to interpret the polls, you need top statisticians with lots of domain knowledge. In order to interpret the betting market, you simply read the odds and are done.
The particular market available to US residents has separate issues about "reading the odds" where events that strictly contain other events are often priced multiple pennies cheaper, the total number of expected presidents is often ~1.1, events with probability at least 0.9 are almost always underpriced, often by a nickel or more, etc.
Add withdrawal fees and you could easily lose money while making an accurate prediction.
On the other if the bet paid 100% of your money you would net (100% * 0.9 - 2%) = 88% and your fee dropped you from 98% to 88% a net loss of 10.2%.
PS: These numbers get much worse if the withdrawal fee includes the principle. Then you would be have a net loss of (110% * 0.9 - 2%) = 3% on the first bet and a net gain of 78% on the second.
To simplify not using money for some time period has an opportunity cost. Either because you could have paid off a loan sooner, or bought a T bill, or whatever. If the bet takes a year then at the end of that year you could either have (principle + opportunity) cost if you don’t make the bet, (principle + winnings) if you make the bet and win, or nothing if you lost. Therefore the cost of making the bet in 2010 that paid out in 2020 isn’t the money you put upfront, but the money you could have had in 2020 without making the bet.
However, the winnings are calculated based on your initial payment not the payment + opportunity cost. So let’s look at a bet that pays 1$ after fees and costs you 1$ worth of opportunity costs. In such a bet the fees reduce your winnings to zero because you could have the same amount of money without the risk of the bet.
The effective withdrawal fee is 1 - (actual gains / ( winnings - opportunity cost)).
So if the actual fee is 10% your winnings are 2$ and your opportunity cost is 1$ them the effective fee is 20%. In other words a 10% withdrawal fee dropped your actual gains from 1$ to 80 cents.
However if your winnings where 1,000$ then your effective withdrawal fee is 10.01% dropping your actual gains from 999$ to 899$. In other words fees are more impactful on high probability bets than low probability bets.
So if you go by the most important metric of what was trying to be predicted, 538 was far more accurate for the one giant data point.
1. There's a statistical gadget specifically for doing this—a "scoring rule" [1] which is a principled way to compare different probabilistic predictions. A bunch of scatterplots of random quantities against each other are... not that.
By comparing only binary win/loss predictions instead of probabilities, like in the first chart, you throw away almost all information contained in the probabilistic estimates—if Democrats win a state, there's no bonus for predicting (say) 95% Dem instead of 55% dem.
It's plausible that 538 would actually win under a proper scoring rule, because betting markets were underconfident (relative to 538) in deep dem/rep states (predicting e.g. <95% Dem win in VT, vs 538's >99%). [2]
2. The calibration analysis assumes that different state win/loss rates are independent, but that's really untrue: 538's predictions were specifically not independent because they assumed polling errors were correlated between states.
3. Many of the other scatterplots look outlier-driven and don't include r^2 or p-values. With so few datapoints, it's unclear if they are meaningful at all.
[1]: https://en.wikipedia.org/wiki/Scoring_rule
[2]: Maybe we should cut prediction markets some slack here because liquidity constraints make them inaccurate for small probabilities. If that's the article's position, though, they should address this instead of just... not using a scoring rule.