News for hackers. Stuff that matters.
Re:The Secret Agenda of a Facebook Quiz by fovc on Sunday January 22, 2017 @07:39AM (#13454931)
And yet for all their magic, they were just as wrong about the outcome of the election:

You might think from a casual reading of the Cambridge Analytica press release that they predicted the outcome of the election. They did not. A company spokesman called reporters before election day to say that Trump had only a 20 per cent chance of winning.

Source: http://www.spectator.co.uk/2016/12/the-british-data-cruncher...

Re:The Secret Agenda of a Facebook Quiz by gareim on Sunday January 22, 2017 @07:44AM (#13454947)
I'm not sure I get this logic. If I told someone they had a 16.6% chance of rolling a die and getting a 6, and then they proceed to roll the die and get a 6, would you laugh and tell me that I was wrong?
Re:The Secret Agenda of a Facebook Quiz by forgetsusername on Sunday January 22, 2017 @08:19AM (#13455066)
I think many people would, yes. Which is frightening.

I just read an anecdote about Sid Meier dealing with this in the Civilization games in Michael Lewis' Undoing Project: people couldn't stand losing battles of 33% 3 times in a row.

Re:The Secret Agenda of a Facebook Quiz by delinka on Sunday January 22, 2017 @09:50AM (#13455365)
That's not a prediction of the outcome. That's a statistic. A prediction of the outcome would be "you won't roll a 6."
Re:The Secret Agenda of a Facebook Quiz by Jarwain on Sunday January 22, 2017 @10:13AM (#13455447)
The article doesn't claim that Cambridge Analytica predicted the outcome. The only factual claim about the outcome made by the company, according to the article, is that statistic.

The article, however, took that statistic and implicated it as a prediction when all it really was was a statistic.

Re:The Secret Agenda of a Facebook Quiz by dagw on Sunday January 22, 2017 @10:31AM (#13455535)
If you asked me to predict whether you will roll a six or not, then I will predict that you won't. Is that a bad prediction?
Re:The Secret Agenda of a Facebook Quiz by delinka on Sunday January 22, 2017 @10:52AM (#13455650)
It's certainly an incorrect prediction if I roll a six. You didn't simply state the odds and leave it at that, you actually made a prediction. Is it a "bad" prediction? Depends on how you're defining "good" vs. "bad" predictions. If the quality of the prediction is not based on the outcome (i.e. whether the prediction is correct), but is instead based on the odds themselves, then that's a conversation that leads to "bad but correct" predictions.
Re:The Secret Agenda of a Facebook Quiz by pavelrub on Sunday January 22, 2017 @08:46AM (#13455156)
If it was your job to measure all the things that affect the roll of a die, and this is the result of your prediction, then I would indeed say that you failed to do your job properly. You could defend the result by claiming that it is impossible to get better measurements, and that the 16.6% chance is a reflection of our unavoidable ignorance of the system, but this doesn't really seem applicable to predicting election results.

What you are essentially saying is that every prediction that doesn't claim certainty is always correct. If I "predict", by looking at a crystal ball, that there is an 80% chance of an asteroid destroying Earth tomorrow, and no asteroid destroys earth, do you think that it makes sense to say that I was correct in my prediction, because I said there was a 20% chance of it not happening? Surely you must agree that there some sense in which my prediction was wrong, or at the very least more wrong than NASA's prediction. The election predictions were wrong in the same sense.

Re:The Secret Agenda of a Facebook Quiz by gareim on Sunday January 22, 2017 @09:01AM (#13455210)
A lot of words to say very little.

>What you are essentially saying is that every prediction that doesn't claim certainty is always correct.

This seems to be the crux of it. No, what I am saying is that when you want to laugh at the polls/pollsters, you should be arguing how their methodology was wrong and what they could have done to find the true proportion +/- some standard error with however much confidence level. Simply laughing at pollsters when the minority wins is not good enough. 20% chance of winning and then actually winning does not seem all too unlikely to any reasonable person.

Re:The Secret Agenda of a Facebook Quiz by pavelrub on Sunday January 22, 2017 @09:33AM (#13455305)
> you should be arguing how their methodology was wrong and what they could have done to find the true proportion +/- some standard error with however much confidence level

There are countless of arguments about how their methodology was wrong and what could be improved, all over the internet. Almost nobody disputes that the models were bad. I thought this part was obvious.

> 20% chance of winning and then actually winning does not seem all too unlikely to any reasonable person.

It seems about 80% unlikely. But the question isn't whether it seems unlikely or not, but whether it makes sense to call the prediction results "wrong". 20% chance of an asteroid not hitting Earth and then not hitting Earth might not seem extremely unlikely, but the prediction that it had an 80% chance of hitting Earth is still a bad prediction.

Re:The Secret Agenda of a Facebook Quiz by Jarwain on Sunday January 22, 2017 @10:22AM (#13455487)
Is it really a bad prediction? Or just a wrong one?

Sure, you could say wrong predictions are bad. However, maybe it was the best prediction given the data available before a given event. Its weird assigning a kinda subjective good/bad to an event when it is just an assigned probability based on what is known. All we can really conclude is that the improbable happened; we didn't have enough data or the right methodologies to make a more accurate prediction. Learn what we can, and apply it to the next event.

Re:The Secret Agenda of a Facebook Quiz by pavelrub on Sunday January 22, 2017 @10:50AM (#13455642)
I would say that predicting the chances of an asteroid hitting Earth based on looking at a crystal ball is a pretty bad prediction.

I agree with your general point. Whether a better prediction could've been made based on available data is really the key question here, and this is exactly what allows us to say that crystal ball predictions are bad.

In the context of predicting election results, I think it is fair to assume that, in principle, there should be enough available data (or an ability to collect such data) to make more accurate predictions. This also seems to be the assumption of all major polling agencies, and was also the assumption in investigating the results of Brexit polls. This is precisely where it differs from dice rolling. It therefore makes sense to assume that the predictions were inaccurate due to methodological reasons, as opposed to pollsters having no practical way of accessing the relevant data.

Re:The Secret Agenda of a Facebook Quiz by xapata on Sunday January 22, 2017 @10:27AM (#13455519)
I'm not sure which ones you read. Nate Silver's analysis seemed pretty sound. His team pointed out that, just as with the housing crisis, if the models were wrong in one state, they were probably wrong in all states in the same way, giving Trump a decent chance at winning.

Note that Taleb criticized the speed that Nate changed estimates, not the estimates themselves.

Re:The Secret Agenda of a Facebook Quiz by dagw on Sunday January 22, 2017 @10:38AM (#13455573)
There are countless of arguments about how their methodology was wrong and what could be improved, all over the internet.

Could you link to a couple of your favourites? Non of the ones I've seen felt very convincing.

Re:The Secret Agenda of a Facebook Quiz by pavelrub on Sunday January 22, 2017 @11:01AM (#13455704)
Sure:

[1] https://fivethirtyeight.com/features/the-polls-missed-trump-...

[2] http://www.pewresearch.org/fact-tank/2016/11/09/why-2016-ele...

While it will take more time to figure out the precise reasons for the failure of the polls, the consensus is that such a failure indeed happen, and there are a number of competing hypotheses for why it happened. Other than on HN, nobody claims that the errors were due to some inherent unpredictability which cannot be addressed through better methodology.

Re:The Secret Agenda of a Facebook Quiz by dagw on Sunday January 22, 2017 @11:11AM (#13455749)
I think everybody agrees that something went wrong and many people think that something should be done (although I personally think that medias inability to interpret and report on polling errors and uncertainty did greatly exasperate the situation). I just haven't seen any good articles making a strong argument about what went wrong and how to fix it
Re:The Secret Agenda of a Facebook Quiz by pavelrub on Sunday January 22, 2017 @11:21AM (#13455790)
I don’t know, I’m under the impression that people here think that winning with a 20% predicted chance of winning isn’t indicative of anything wrong with the polls, because it’s like predicting the outcome of a die, and therefore nothing should be done at all. This false analogy is precisely what I’m trying to disprove.

I agree that there are currently no strong arguments about what were wrong and how to fix it, but I think it's because it takes time to investigate those things. According to Pew, the American Association for Public Opinion Research has a committee investigating it, and they should release their report in May. It took a 6-months investigation to come up with a report about what went wrong with the Brexit polls, so it will probably take a similar amount of time with this.

Re:The Secret Agenda of a Facebook Quiz by dagw on Sunday January 22, 2017 @04:25PM (#13457587)
You're conflating the polls with the predictive models. The polls never said Trump had a 20% chance of winning because the is not how polls work.

There where a dozen or so models which took the polls as input (some added other inputs as well) and produced a probability of a candidate winning. Some of those models where garbage (Huffington post I had Trump at ~1.5-2%) and some where pretty good (fivethirtyeight had Trump at ~30% and trending upwards). The question about how you should interpret these numbers is a more open one. What does it mean to give numeric probabilities to events which are completely unique and will only occur once (cue discussion of Bayesian vs Frequentist inference here).

Admittedly the question about if the model was right or wrong is difficult to disentangle from the question about bad polling. No model can work correctly if you feed it garbage data. The only criticism you could make is that they should have been even more critical of data they where getting from certain polls than they where.

Now as to was there something wrong with the polls? Obviously. But the interesting question is what went wrong. They where pretty good at forecasting the national popular vote, while at the same time getting certain mid-western swing states dramatically wrong. So there is obviously something in their methodology which seems to works fine when looking at the country but fails when looking at certain states.

But like you I look forward to seeing more detailed investigations coming out in the next few month.

Re:The Secret Agenda of a Facebook Quiz by arrel on Sunday January 22, 2017 @11:31AM (#13455846)
The difference is that the roll of a die is random, but the outcome of an election gets more deterministic as it approaches. Election prediction is figuring out what people have already decided they are going to do.
Re:The Secret Agenda of a Facebook Quiz by BigJeffeRonaldo on Sunday January 22, 2017 @03:09PM (#13457100)
Well, the model at least does not maximize likelihood
Re:The Secret Agenda of a Facebook Quiz by akjainaj on Sunday January 22, 2017 @08:06AM (#13455007)
I get your point but that's not a fair comparison because there were 2 choices, not 6.
Re:The Secret Agenda of a Facebook Quiz by SixSigma on Sunday January 22, 2017 @08:19AM (#13455065)
Ok. If I told you you had 80% chance of flipping a coin to a head and you got a tail, would you say I couldn't predict things ?
Re:The Secret Agenda of a Facebook Quiz by dagw on Sunday January 22, 2017 @10:34AM (#13455553)
6 and Not 6 are two choices
Re:The Secret Agenda of a Facebook Quiz by endless1234 on Sunday January 22, 2017 @08:29AM (#13455097)
How does Trump winning prove he had more than a 20% chance of winning?
Re:The Secret Agenda of a Facebook Quiz by delinka on Sunday January 22, 2017 @09:52AM (#13455369)
I feel this is the rub with this particular comment thread. If all they gave were stats, then they didn't make a prediction.
Re:The Secret Agenda of a Facebook Quiz by forgetsusername on Sunday January 22, 2017 @08:18AM (#13455058)
Analysis is useless if it isn't right each and very time, got it.