Re:Nassim Talebs case against Nate Silver is bad math
by andrewla on Wednesday April 10, 2019 @10:55AM
(#19625298)
I confess, I still don't follow the critique. Let's take the output of a skin-in-the-game model, like a binary futures market. This also predicts numbers between 0 and 1, that look a lot like probabilities, but by the argument presented here, they are not probabilities, and if you look at the output of a futures market, then you are being scammed because the market itself is not giving a decision boundary; it's just outputting the last trade price.
Yes, this allows a forecaster to say that even if the <50% candidate wins, that they weren't wrong, but so what? The forecasts should be reasonably well-calibrated -- that is, of all 20% predictions, 20% of them go in the 20% direction. 538 recently published a report of their calibration, which bears out their results (and kicked off the latest round of name-calling between Taleb and Silver).
Is the criticism just that we shouldn't look at Silver as a person with anything interesting to say because he won't commit 100% to one side of an argument? I honestly don't understand what the critique is, and this link I don't find helpful.
A piece by Isaac Faber [1] from December last year clearly described what Taleb's position is. A summary paragraph towards the end:
> Here we can say, with some confidence that FiveThirtyEight predictions are not always reliable probabilities. However, they masquerade as one, being between 0 and 1 and all. This is Taleb’s primary argument; FiveThirtyEight’s predictions do not behave like probabilities that incorporate all uncertainty and should not be passed off as them.
Faber's piece was discussed on HN [2] soon after it appeared.
[1] https://towardsdatascience.com/why-you-should-care-about-the...
[2] https://news.ycombinator.com/item?id=18704376