News for hackers. Stuff that matters.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by WheatMillington on Tuesday April 25, 2023 @07:16PM (#35707989)
Nate Silver is a guy who got lucky one year, and managed to convince everyone he's a genius.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by saghm on Tuesday April 25, 2023 @07:22PM (#35708030)
Not making a judgment about his methods or not, but he was pretty accurate more than once; in 2008 he predicted 49/50 states correctly in the US presidential election, and in 2012 he predicted all 50 correctly. You might be right that he isn't as smart as people think, but "got lucky one year" is an exaggeration.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by keithwinstein on Tuesday April 25, 2023 @08:04PM (#35708303)
I'm a big fan of Nate Silver, but I think this idea that he "predicted N/50 states correctly" in 2008 and 2012 is mostly a myth.

(1) The website doesn't "predict" a state when it reports that the probability of (e.g.) an Obama victory in Florida is 50.3%, which I believe is what 538 assessed in 2012. If you think that a state's probability distribution is 50/50 between Candidate X and Y, and the outcome is Candidate X, obviously you don't get any points for a good prediction. You didn't make a prediction. And if you say it's 50.3% probability, it's essentially the same situation. You didn't make (much of) a prediction. Which is okay!

(2) In terms of Brier Scores or entropy scores, which compare the probability with the outcome and give more "points" the stronger the prediction was made (vs. what actually happened), FiveThirtyEight's predictions are generally just middle-of-the-pack compared with other websites and mechanisms of aggregating these polls. (See, e.g., https://web.math.princeton.edu/~sswang/Wang_Origins_of_Presi...) There's no strong evidence that 538's predictions are much better or worse than other reasonable ways to weight and aggregate the recent polls that anybody else might come up with.

(3) And... EVERYTHING we say that tries to quantify the "accuracy" of the predictions is only talking about the state of the 538 website ON THE MORNING OF THE ELECTION. But that's probably the LEAST useful day to be looking at FiveThirtyEight. By then, you can just wait a few hours and get the actual results. :-) The predictions that are really valuable are the ones that are made some time in advance, e.g. one or two months.

But (a) it's hard to directly measure the "accuracy" of a prediction like that, because... things change and we can't rerun the universe, and (b) it's not like the model is somehow time-invariant to be able to measure the accuracy indirectly by proxy; there are various "time until election" factors included in these model itself, the dynamics of the election itself probably have a "time until election" dependency, AND the website operators, including 538, are generally manually tweaking the model parameters or behavior in ways they don't always disclose as the election evolves.

So basically... it's hard to say. I think 538 has done great work, and in general it's incredibly useful to have a consistent world model that you can rationally update to discuss the effect of recent news. The FiveThirtyEight political models are as good as any for that! But I don't think there's much strong evidence they're better.

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by saghm on Tuesday April 25, 2023 @08:10PM (#35708325)
That's fair! I probably should have realized that, given that I already knew that his 70-30 prediction of Clinton winning in 2016 didn't mean he "predicting wrong"; I'm guessing I was just less aware of how his models worked back in 2012 than I was by 2016, so I didn't really consider what it meant to "predict the states".
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by yowzadave on Tuesday April 25, 2023 @11:37PM (#35709425)
I think many people subconsciously conflate a "70% chance of winning" with a "70% of the vote" prediction, when the reality is much closer--a 70% chance of winning could very easily go the other way, given some systemic polling error and/or late-breaking news!
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by saghm on Wednesday April 26, 2023 @12:22AM (#35709610)
Yeah, it's definitely tricky to communicate. The chances of _either_ candidate getting 70% of the vote in any recent presidential election would be incredibly small; even 55% would be a huge anomaly. Most people wouldn't feel very enlightened looking at a probability distribution of potential vote percentages for each candidate on a CNN graphic though, especially when they'd basically just look like giant spikes around 48-52%.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by dragonwriter on Tuesday April 25, 2023 @10:09PM (#35709005)
> But it’s hard to directly measure the “accuracy” of a prediction like that, because… things change and we can’t rerun the universe

Its hard to measure the accuracy of a prediction like that, but with a big collection of predictions (and since every cycle Silver predicts a lot of individual elections – not just each of the state elections for presidential electors, including both the statewide electors and the by-district electors where those exist, but also house, senate, gubernatorial, and some other races – each election cycle, this exists) you can bucket them in similar probability buckets and say, for instance, and see how often things Silver predicted in each probability bucket came true vs. the probability range of the bucket, and get a good idea of the accuracy of the predictions.

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by foxandmouse on Tuesday April 25, 2023 @07:27PM (#35708060)
At FiveThirtyEight, their goal is to deliver high-quality quantitative analysis by utilizing record-level data. It's ironic that the type of comment you made is typically found on sports news websites that lacks the rigor and depth of a quality analysis.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by test098 on Tuesday April 25, 2023 @07:25PM (#35708048)
> Silver was named one of The World's 100 Most Influential People by Time in 2009 after an election forecasting system he developed successfully predicted the outcomes in forty-nine of the fifty states in the 2008 U.S. presidential election.[2] His subsequent election forecasting systems accurately predicted the outcome of the 2012 and 2020 presidential elections with a high degree of accuracy, and - at the time - gave Presidential candidate Donald Trump the highest chance of success the day of the 2016 presidential election of reputable polls at the time.

- https://en.wikipedia.org/wiki/Nate_Silver

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by dragonwriter on Tuesday April 25, 2023 @07:51PM (#35708227)
> gave Presidential candidate Donald Trump the highest chance of success the day of the 2016 presidential election of reputable polls at the time

This makes a fundamental error of confusing “polls” (which is not a category Silver’s work falls into) with statistical predictions based on aggregates of polls (which is what it is.)

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by itsdrewmiller on Tuesday April 25, 2023 @08:43PM (#35708514)
I think it's poorly worded - there were a bunch of other aggregators at the time who said things like Clinton was 99% likely to win. IIRC 538 assumed that polling bias could be correlated between states while most analyses were assuming that each poll was completely independent, but that's probably an oversimplification given the vagaries of memory.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by dllthomas on Wednesday April 26, 2023 @11:02AM (#35714763)
Your recollection matches mine, fwiw.
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by dragonwriter on Tuesday April 25, 2023 @07:50PM (#35708211)
> Nate Silver is a guy who got lucky one year, and managed to convince everyone he’s a genius.

Nate Silver is a guy who is very good at applying statistical modelling to a number of domains (including election analysis, where he basically revolutionized the field with everyone else doing poll-aggregate-based-prediction – which no one was previously – following in his footsteps, but mostly doing a worse job), but who morphed into another generic center-right Democratic political talking head. Whether that’s a structural demand of Disney that will go away now that he is leaving Disney or whether its what he feels his groove is now…will be interesting to see.

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by mardifoufs on Wednesday April 26, 2023 @01:37AM (#35709940)
Genuine question, since I dont really follow him outside of elections, but in what way is he center right? I'm not american and I'm an immigrant/poc so I'm not especially biaised to the right but he always came off to me as pretty centrist and "liberal". When did that change?

(The 2020 election was the last time I really heard about him, and the most controversial takes he had was some very mild reservations on lockdowns and school closures... at a time when the entire world reopened schools except for the USA, which is why it didn't register like a hot/right wing take at all to me.)

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by cyanydeez on Tuesday April 25, 2023 @08:39PM (#35708496)
He didn't get lucky.

He found a trend and published it widely.

He changed the very thing he was studying. Because politics was already a statistical game, his original models failed to reproduce.

He clearly published a single result.

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by greesil on Tuesday April 25, 2023 @09:10PM (#35708690)
His approach is pretty sound, and straightforward if you listen to his podcast. Proper treatment of fat tails via student t distributions and correlated variables with good (subjective) judgement on polling quality.

I find that most people with negative views of Nate's approach are innumerate with regards to statistics. They don't criticize the model because they can't be bothered to understand it, or even basic probability, and certainly don't rank his performance vs other modelers which wouldn't require understanding the model.

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by petulla on Tuesday April 25, 2023 @10:25PM (#35709083)
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by greesil on Tuesday April 25, 2023 @10:34PM (#35709127)
What are the consequences of this, in terms of prediction accuracy?
Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by greesil on Wednesday April 26, 2023 @02:29AM (#35710181)
so basically what TFA says is if you build a new model on Nate's model outputs and use that to compute conditional probabilites of various improbable scenarios you see weird shit. State level polling data according to Nate isn't all that great (read: high uncertainty, moves around when a new poll comes out) unless it's a swing state. But hey when you go looking into noise you are going to see.... weird shit.

Did you Google any other good blog posts?

Re:Disney is shrinking FiveThirtyEight and Nate Silver (and ... by trc001 on Tuesday May 02, 2023 @07:57PM (#35795931)
I’ve been saying some version of this to people for years.

But in his defense, I think he’s been more the victim of the secular decline in polling quality than anything else.