News for hackers. Stuff that matters.
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @10:15AM (#19757716)
They obviously got 16 very wrong. However, what is even more concerning is that Nate lately became a political hack, doing the ugly work for the democratic establishment. Just look at his twitter you can clearly see it. So I would take any "data science" he does with a grain of salt.

Edit: An example of his tweets:

"It's probably worth noting that while this group, Justice Democrats, calls Biden "out-of-touch" with the "center of energy" in the Democratic Party, only 26 of the 79 candidates it endorsed last year won their primaries, and only 7 of those went on to win the general election."

Why didn’t he add the statistic that Biden already ran 2 times in the past and won 0 times.

Re:How Good Are FiveThirtyEight Forecasts? by pwinnski on Friday April 26, 2019 @10:25AM (#19757834)
They did not, he has not, and one should not any more than one always should.

538's forecast was easily the most accurate for November 2016, by a decent margin. The majority of critics of 538 are "sore winners" who delight in their side having won despite being given a less than 50% chance, but as Silver pointed out at the time: his chances were similar to those in Russian Roulette, and most people consider that too risky to play.

Re:How Good Are FiveThirtyEight Forecasts? by paulgb on Friday April 26, 2019 @10:27AM (#19757843)
Right. It's like saying "Ha! You were wrong! You said there was a 83% chance I wouldn't roll a six, and I did"
Re:How Good Are FiveThirtyEight Forecasts? by insickness on Friday April 26, 2019 @10:34AM (#19757930)
More like saying, "We said there was a 17% chance that it could go the other way, so no matter what happens, we aren't wrong."
Re:How Good Are FiveThirtyEight Forecasts? by b_tterc_p on Friday April 26, 2019 @10:46AM (#19758063)
This is disingenuous. A prediction is a prediction. There is a world of difference between 51% certain and 99% certain. People excited that their data science models get good predictions on their test set without considering, say, the RoC curve are setting themselves up for failure.

If a model says 75% chance of X rather than Y, that doesn’t mean you should expect X. Depending on context, you might be better off flagging it as unclear or even safer to conclude a possible Y.

Testing calibration of many models across many different trials as a test of data scientist competence is a little strange, but if you want to answer the question of how valuable the predicted likelihood’s are, it’s a decent measure.

Re:How Good Are FiveThirtyEight Forecasts? by Symmetry on Friday April 26, 2019 @10:41AM (#19758004)
That's why it's important to do what 538 did here. Look at all the different places where they made 5 in 6 predictions ( and there are a lot of different races) and make sure that 5 in 6 of them happened.
Re:How Good Are FiveThirtyEight Forecasts? by protomyth on Friday April 26, 2019 @10:34AM (#19757933)
538's forecast was easily the most accurate for November 2016, by a decent margin.

No, Investor’s Business Daily was the most accurate.

Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @10:39AM (#19757989)
If thats your understanding of making a prediction then he was accurate by your standards. You clearly did not watch the election night predictions. NYT live model was miles better than 538s.

https://fivethirtyeight.com/features/how-i-acted-like-a-pund...

Re:How Good Are FiveThirtyEight Forecasts? by novia on Friday April 26, 2019 @11:10AM (#19758311)
Your link is from 2016 before the presidential election but after the primaries. Nate is admitting that he underestimated Trump in the primaries and talks about what happened. This does not seem particularly relevant to the subject of election night predictions.
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @11:19AM (#19758420)
This is regarding his bias. If anything the link is even more relevant. He repeated the similiar mistakes.
Re:How Good Are FiveThirtyEight Forecasts? by pwinnski on Friday April 26, 2019 @11:25AM (#19758485)
His main mistake as highlighted in that article is that they didn't use statistics for the nomination process, but guesswork. For the election proper, they did use statistics. So no, they did not repeat the mistake at all.
Re:How Good Are FiveThirtyEight Forecasts? by notheruser on Friday April 26, 2019 @11:40AM (#19758656)
His election night model wasn't good, but he isn't known and respected for his election night model, so who cares?
Re:How Good Are FiveThirtyEight Forecasts? by paulgb on Friday April 26, 2019 @10:25AM (#19757832)
I have seen his Twitter feed, do you have any specific examples? Obviously he's talking a lot about Democrats right now more than Republicans because Trump is the presumed Republican nominee, but covering the race isn't "doing ugly work for the Democratic establishment."
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @10:34AM (#19757932)
"It's probably worth noting that while this group, Justice Democrats, calls Biden "out-of-touch" with the "center of energy" in the Democratic Party, only 26 of the 79 candidates it endorsed last year won their primaries, and only 7 of those went on to win the general election."

Please explain to me what is the point of such a tweet.

Re:How Good Are FiveThirtyEight Forecasts? by paulgb on Friday April 26, 2019 @10:49AM (#19758098)
It contextualizes a remark from a group that claims to speak for the party's electorate by providing hard numbers. That's what people who follow Nate expect from him on Twitter.

Edit: to be fair, if memory serves JD mostly/entirely endorsed primary challenges rather than incumbents. The base rate for primary challengers is pretty low, so ~1/3 is a respectable win rate and I would concede that it's a little disingenuous not to mention that.

Re:How Good Are FiveThirtyEight Forecasts? by dragonwriter on Friday April 26, 2019 @10:59AM (#19758193)
The group doesn't claim to speak for the part’s electorate. It claims to speak for a particular ideological faction, toward which it also claims the party electorate has moved since the last election, so that a candidate representing the faction that was uncontroversially still dominant in 2016 is now out of line with the dominant view in the party.

Nate seems to be implicitly arguing out that that claim would not have been inaccurate if made in 2016 (and further that the claim that may have been implied but was not made that the same faction is positioned for success in the general election should it succeed in the party nominating contests would also.have been untrue if made in 2016.) But that is silly, since it's a claim about how the 2020 landscape is difft than 2016.

Or, maybe Nate is conflating endorsements with predictions rather than advocacy, and trying to argue that the rate of success of the groups endorsements is somehow indicative of the reliability of their descriptive statements.

Or maybe Nate, who aside from running a prediction site is also an activist aligned with the faction being criticized against the one doing the criticism, is not posing a rational argument at all, just trying to provoke an emotional response by posting something that vaguely insinuates that the criticizers are wrong in some way.

Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @02:29PM (#19760503)
Eloquent writing definitely helps explain my point better. Thanks.
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @11:01AM (#19758214)
This does not contextualize anything in any way. He is selectively picking data to form a narrative. I dont think you are objective at all so I will stop trying to convince you but lastly think about this:

Why doesnt Nate reply to Neera Tanden, CAP or Nancy Pelosi with “stats” on medicare for all support which is polling at 72%?

Re:How Good Are FiveThirtyEight Forecasts? by paulgb on Friday April 26, 2019 @11:32AM (#19758550)
> I dont think you are objective

If you're implying I'm pro-Biden, for the record, he isn't even in my top three. I don't read Nate because he supports my beliefs, I read him because I think he has a good-faith interest in accurate predictions (even if it is impossible for a human being to be 100% objective)

Re:How Good Are FiveThirtyEight Forecasts? by the_watcher on Friday April 26, 2019 @11:06AM (#19758267)
He forecasts elections, and his models include endorsements. This tweet discusses a group that issues endorsements, their stance on a candidate who just entered a primary, and the win rate of candidates endorsed by said group. This is essentially a perfect tweet illustrating something that a forecaster would do.
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @11:54AM (#19758825)
I still expect him to tweet to Biden saying he ran 2 times and won 0 times.
Re:How Good Are FiveThirtyEight Forecasts? by the_watcher on Friday April 26, 2019 @12:35PM (#19759303)
The fact that he hasn't tweeted about every possible statistic is irrelevant to your ask about "the point" of the referenced tweet.
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @12:48PM (#19759439)
Aahh still going over your head. The whole point I am trying to make is that anybody who is fully objective and data driven would refrain from making such statements online, especially when it is uncalled for. Nate has been consistently taking a pro-establishment stance even when he was proven wrong over and over again. To say it in another way he could have said: Justice Democrats, founded in 2017, went from having zero members in the House to 7 within a year. Including AOC, one of the most popular politicians in the US.
Re:How Good Are FiveThirtyEight Forecasts? by the_watcher on Friday April 26, 2019 @01:08PM (#19759646)
It's not going over my head. You support Justice Democrats.
Re:How Good Are FiveThirtyEight Forecasts? by rustyboy on Friday April 26, 2019 @10:50AM (#19758107)
That statistically the fringe wing's candidates don't win?
Re:How Good Are FiveThirtyEight Forecasts? by objektif on Friday April 26, 2019 @10:57AM (#19758176)
Fringe!!! You would name Bernie and AOC and politicians with similiar ideology fringe candidates? This is trying to form a narrative by nitpicking data.