How accurate the "best models" were isn't really relevant to the accuracy of the predictions of experts as a whole. The media was running that Clinton had a 98% chance of winning the morning of the race.
Only if you include random talking heads as experts. Then you get that 98% chance of Clinton winning result. But you can select actual experts to rely on before the outcome happens. The 538 forecast was extremely accurate the previous election and had the 30% outcome the day before. That's actually comparable to a poll of economists on bitcoin.
And to make it even more useful the polling of economists should ask them to give a probability instead of yes/no. It may be true that 96% say "yes it's in a bubble" and the average probability between them of Bitcoin going to 1M$ is actually 20% or more.
Nate Silver wrote at length about this but the gist is that the polls were pretty accurate but some people talked about them imprecisely:
https://fivethirtyeight.com/features/the-real-story-of-2016/
The data gave her 70%. We're in the 30% of universes where Trump won.
In this scenario, you might argue, these are just talking heads. But they are not being asked for a view in order to look popular on TV, they're looking at data and interpreting it in a way that they are happy to be professionally accountable for.
Wisdom of crowds in the context of talking heads shooting the odds on a media out is not going to be as accurate as wisdom of crowds for economists being asked about economic issues.
So, you know, be careful where you're drawing lines here.
Trump winning doesn't disprove that. The models could be correct and we wound up in the 2%.
The chances of me winning the Powerball are one in a hundred million, but me winning doesn't disprove those odds at all.
Tomorrow: I lost money. I guess I just fell into the 2%.
"So back to the mechanism of the model, Taleb imposes a no-arbitrage condition (borrowed from options pricing) to impose time-varying consistency on the Brier score. This is a similar concept to financial options, where you can go bankrupt or make money even before the final event. In Taleb's world, if a guy like Nate Silver is creating forecasts that are varying largely over time prior to the election, this suggests he hasn't put any time dynamic constraints on his model."
http://lesswrong.com/lw/o5x/nassim_taleb_on_election_forecas...
Anyone who proposed 98% certainty was ridiculous and should not be taken seriously either (there could be exceptions but I don't think so). It amounts to beating up a strawman.
But I just don't understand what that quoted paragraph means and how it proves that it was 50% or anything else. A forecast should go bankrupt before it gets to the final event, what? Dynamic time constraints meaning what exactly? I'm genuinely curious.
Maybe Taleb is just an intellectual god that my puny intellect will never completely understand, and I should just take him at his word, and I'm sure he would agree. But I remain skeptical of people who display open contempt for people (I used to follow him on Twitter) who require a better explanation to understand what they're saying.
Can go bankrupt, not should.
He is saying that if in finance someone would make such a ridiculous forecast, his forecast would be shorted into oblivion and he would have lost most of the money bet on the forecast before the event even took place.
Imagine two relatively matched football teams, but a bookie offering 10:1 bets on one of them. There would have been massive bets against that bookie.
Given the recent history of american elections and other factors, a 50:50 prediction is much more sensible than a 98:2 one. But because journalists don't actually put money on their predictions, they can spout whatever numbers.
He is also complaining that even with a lot of polls hovering around 50%-50%, that 98%-2% prediction was still on, which doesn't really make any (math) sense (binary options, delta, ...).
In his math paper he describes this in more precise form.
If I make a bad bet I don't go "bankrupt" or lose that money before the event happens. There's more than one way to structure betting on events than the way he proposes as well, so I still don't understand why his is an authoritative take.
"Given the recent history ... and other factors" is more of an explanation for why someone thought it was a coinflip, but doesn't represent "a proper statistical analysis" modeling 50:50.
Be very careful with those conclusions. In both cases the predictions were that the elections were very close. Trump winning the presidency when the best models we had gave him a 30% chance is not a failing of prediction at all.