News for hackers. Stuff that matters.
Re:Facebook Must Really Suck At Machine Learning by tomp on Friday November 18, 2016 @07:24PM (#12991151)
Funny you mention Breitbart, it's one of the rare media that didn't predict a landslide victory for Hillary.
Re:Facebook Must Really Suck At Machine Learning by notheguyouthink on Friday November 18, 2016 @07:28PM (#12991168)
I don't have a dog in this show, but wouldn't that be quite reasonable/expected if they are skewed towards Republican/Trump?
Re:Facebook Must Really Suck At Machine Learning by snewk on Friday November 18, 2016 @07:34PM (#12991191)
i think the bigger question in wake of hillary's missing landslide, is whether facebook may actually be classifying CNN et. al. as 'fake' news.
Re:Facebook Must Really Suck At Machine Learning by serge2k on Friday November 18, 2016 @07:59PM (#12991344)
Poor polling data doesn't mean you are publishing fake news.
Re:Facebook Must Really Suck At Machine Learning by tomp on Friday November 18, 2016 @07:38PM (#12991211)
A better example would be NYT or WSJ, as CNN is obviously fake news [1].

[1] https://www.washingtonpost.com/news/volokh-conspiracy/wp/201...

Re:Facebook Must Really Suck At Machine Learning by tomp on Friday November 18, 2016 @07:40PM (#12991221)
Not according to my logic, no. If I wanted a candidate to win, I'd imagine it would be better for me to propagandize predictions that my candidate would lose, as to motivate more of his/her supporters to actually vote.
Re:Facebook Must Really Suck At Machine Learning by thesimpsons1022 on Friday November 18, 2016 @07:43PM (#12991248)
according to your logic then the MSM is all biased for trump and breitbart is biased for hillary. sounds legit.
Re:Facebook Must Really Suck At Machine Learning by cderwin on Friday November 18, 2016 @09:09PM (#12991609)
I don't agree with tomp's reasoning but you are confusing cause and effect. If tomp is correct, it only implies that a news outlet biased towards one candidate would predict a victory for the other, not that every media outlet predicts a victory a candidate they are biased against.
Re:Facebook Must Really Suck At Machine Learning by candiodari on Friday November 18, 2016 @10:21PM (#12991894)
Because of extreme amounts of arrogance on the democratic side, this is actually what happened. It's the same as you hear from Russians. If something's in any big newspaper, it is

(a) fake

(b) it is rather obvious what the truth is

I'm not saying this is the level the MSM has dropped to, but it most certainly is where it's going.

I'd also like to say that this is what's happening to large media worldwide, and is in fact being used to marginalize the victims of the current economic climate.

Exactly what leftists SHOULD NOT do, if anything they should err on the side of the poor.

http://edition.cnn.com/2016/09/14/opinions/america-doing-bet... (and 1000s other articles)

vs

http://www.wsj.com/articles/election-2016-is-propelled-by-th...

and the "fake news" version of it

https://www.videoblocks.com/video/detroit-ghetto-driving-1-d...

Now read the Hillary articles again, assuming you 100% believe the above links and what they mean about the MSM. I think you'll understand better what is happening.

Frankly the MSM lies about economic facts that are very much in the face of a hell of a lot of people, and that causes people to not trust them anymore. This seems natural, and ... censorship of fake news will make this worse, not better (or better put: I have enough Russian friends and stories from them certainly indicate it will).

But as someone who regularly visits Middle/Eastern Europe but does not live there I see the same thing. I read the newspapers and ... I just don't believe them. The difference with the country I see when visiting is too great. Online fora, filled with locals are easy to find and while filtering absolutely is a necessity, they are generally telling the truth. There are areas where it is worse, e.g. immigrants (and whether or not a particular town was seriously damaged/looted during a "protest" march), but it is certainly true in general. And yes, these fora do not put forward the view the American left has of the world. They don't like immigrants (too many stories of looting, extremely poor work ethic, attacks, criminal behavior, ... combined with stories of the government working against the victims. It is extremely hard to not believe all of them when you've had 2 friends tell you one. In confidence, because they were afraid of retaliation at work.

I think that's one thing people don't realize. The main fake news, in terms of reach, are sites like reddit, and it's country-local alternatives.

But the story goes like this: they had rented a "work" apartment, a place to sleep close to work for during the week, avoiding a commute after a long day at work. Work is in the middle of the capital city, surrounded by immigrant neighborhoods, so this apartment was in an immigrant neighborhood. They got constantly treated very unfriendly in the hallways, but that, they can take. One day a few obviously muslim youth stormed in, speaking French, screaming to eachother how it's their right to rob kuffar (arabic word meaning infidel in the sense of non-muslim, or cockroach), lots of screaming, obviously high. They broke the tv they were stealing, and beat my friend and his colleague in revenge for that. They had guns, took several macbooks, their cell phones and wallets, beat them up, and took their time to smash up the place. At the hospital they could not see a police officer, they knew where the police office was in the hospital, but there was no-one there that wanted to see them, so they went to the police station 2 days later. The police refused to take down a complaint ... They reported it at work and a director called them in that they were not to talk about it or they'd "face consequences", as those assholes say. Why ? Because nothing good could come of this. Either the story dies and they'll be frustrated, or the story gets attention ... and, the director felt, city hall would be mad, and there might even be an attack on the hospital in revenge for the story. And just so we're clear, I know these people. This happened, in the sense that I know one of the 2 people this happened to.

Now this is one data point, and I get it, it's not like crime is new to that part of town (violent crime like that is, though, pickpocketing is what it used to be). It's the reaction to it that makes me believe a lot of far right nuttery. If you read the newspapers, you'd think this was a harmonious neighborhood with everyone living together peacefully. Regular initiatives and public displays about how integrated everyone is, but, and this is important, a drive through it will make it incredibly obvious that there is exactly one ethnic group living there, or if others are living there, they're certainly not willing to show their faces on the street. The neighborhood has a reputation that women cannot go there and you should, under no circumstances, take your girlfriend there, but as a man you can eat there, eat well, and cheap (and there's lots of businesses where one might work nearby, including a huge hospital), but don't take expensive phones or a wallet with either your ID or more money than you'd need to eat. There are no businesses, not a single one, that is owned by anyone from even a slightly different ethnic group.

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @07:50PM (#12991286)
Never followed or watched Breitbart, but CNN and friends presented it as a poll, presumably done with rigor and impartiality. Either they couldn't afford to pay their statisticans or they deliberately fudged them and lied.
Re:Facebook Must Really Suck At Machine Learning by BinaryIdiot on Friday November 18, 2016 @08:14PM (#12991404)
> Either they couldn't afford to pay their statisticans or they deliberately fudged them and lied.

So it's only those two options? It's not possible that the polling was just flawed or wrong but it's they can't pay them (implying they have shit statisticians) or they lied?

Not a fan of most of the media but come on there are more options here. It was a tough election to predict. Bringing in two, fairly unpopular candidates is going to create some interesting results.

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @09:47PM (#12991765)
> So it's only those two options?

The implication was that they fudged their data to make a candidate look better. Clearly they have the money to pay for the poll and statisticians.

The more I think about the more it is seems one of the reason for the loss was this overconfidence. News channels were often way too happy to put up a map with Hillary in the 400+ points and the whole map being blue. They believed that would just discourage those voting for Trump to show up. In their it was a terrible blunder (or win depending who you support), in the end more people who would have voted for the Democrats stayed home.

Re:Facebook Must Really Suck At Machine Learning by hga on Saturday November 19, 2016 @06:55AM (#12993065)
Overconfidence is certainly a story we're getting out of the Clinton campaign; WRT to the Rust Belt where she decisively lost the election:

Bill's advice to put effort into was ridiculed.

Russ Feingold, who was as far as I know honestly expected to win his election for Senator (note he'd just lost the seat 6 years earlier in the off-season 2010 Republican langslide), desperately asked for help towards the end of the campaign and was ignored.

It's been reported that a computer system named Ada was driving pretty much all these sorts of decisions, and it said the Rust Belt was safe.

And there are some other things I think I'm forgetting this early in the morning, anyway, just how wrong this was is demonstrated by how close Minnesota ended up. If Team Trump had realized the opportunity there earlier and put more resources into it, he might well have won it. That that Democratic (well, DFL) bastion was in play....

Re:Facebook Must Really Suck At Machine Learning by nl on Saturday November 19, 2016 @07:34AM (#12993211)
Or their models were bad?

Take a look at the open source election models. They all make similar mistakes in that they don't properly propergate sources of errors from polls into their prediction. 538 did a good job but was criticised for it.

I don't think you need a conspiracy when simple incompetent will do.

Re:Facebook Must Really Suck At Machine Learning by opsiprogram on Friday November 18, 2016 @07:59PM (#12991341)
tomp maybe Breitbart got brought up for content like this: http://www.breitbart.com/tech/2015/12/08/birth-control-makes...

It's mostly unsubstantiated claims, without scientific rigor to back them up (ex: they cite papers but misrepresent the findings), and heavily mixed with personal opinion. Breitbart may have been right about the election, but that doesn't really seem to be because the website is dedicated to clear objective journalism... Wouldn't you agree?

Re:Facebook Must Really Suck At Machine Learning by tomp on Friday November 18, 2016 @08:08PM (#12991373)
> It's mostly unsubstantiated claims, without scientific rigor to back them up (ex: they cite papers but misrepresent the findings), and heavily mixed with personal opinion.

Sounds like pretty much any non-scientific media citing scientific articles.

Re:Facebook Must Really Suck At Machine Learning by opsiprogram on Friday November 18, 2016 @08:12PM (#12991395)
Okay, so other news sources also can be biased/wrong/misrepresent science ... how does that make this Breitbart article more accurate? or Breitbart a more objective news source?
Re:Facebook Must Really Suck At Machine Learning by tomp on Friday November 18, 2016 @08:15PM (#12991417)
Nah, same bullshit (neither more nor less fake), just in the other direction. Best to read both left-ist and right-ist media to avoid bias. Although personally I sometimes find Breitbart more agreeable, as they seem opposed to political correctness and unafraid of voicing inconvenient truths (opinions?).
Re:Facebook Must Really Suck At Machine Learning by hga on Saturday November 19, 2016 @06:59AM (#12993085)
It's Milo being Milo, which any regular reader of Breitbart, or anyone sensitive to nuance and style, would realize is more opinion than official un/less biased "news".
Re:Facebook Must Really Suck At Machine Learning by morgante on Friday November 18, 2016 @07:48PM (#12991269)
Incorrectly forecasting an election result (where polls ended up having substantial errors) is hardly "fake" news.
Re:Facebook Must Really Suck At Machine Learning by tomp on Friday November 18, 2016 @08:11PM (#12991391)
Presenting the election result as 98% certainty is either fake or insanity (for context, that would mean (on average) only 1 wrong prediction in 200 years of US presidential elections).

Edit: admittedly, that was Huffington Post, hardly a standard for good journalism. E.g. NYT had 80%, 1 wrong in 20 years, which is still quite optimistic IMO.

Re:Facebook Must Really Suck At Machine Learning by dllthomas on Saturday November 19, 2016 @01:12PM (#12994692)
Isn't 98% certainty "1 wrong in 50 years", and 80% "1 wrong in 5 years"?
Re:Facebook Must Really Suck At Machine Learning by tomp on Saturday November 19, 2016 @08:41PM (#12996801)
No, it's "1 wrong out of 50 events", but elections are only every 4 years.
Re:Facebook Must Really Suck At Machine Learning by dllthomas on Saturday November 19, 2016 @11:00PM (#12997197)
Ah, that's what I was missing. That said, regardless of how often these events are occurring, we shouldn't be all that surprised by a 1/5 event.
Re:Facebook Must Really Suck At Machine Learning by dllthomas on Sunday November 20, 2016 @03:40PM (#13001078)
Thinking further about this, I think framing it in years is highly misleading. If we imagine an event that occurs once every ten-thousand years, with even probability of two outcomes, then if I pick one of the two as my guess and I correctly assign a 50% probability, you can say "you're saying your guess will only be wrong, on average, once every twenty-thousand years!" That's technically true, but I'm clearly not making too bold a claim!
Re:Facebook Must Really Suck At Machine Learning by tomp on Sunday November 20, 2016 @05:40PM (#13001716)
Yeah I sometimes question the sensibility of even trying to assign probability to what are essentially one-time events. I prefer to think of it, the probability is either 0% or 100%, we just don't know it yet!
Re:Facebook Must Really Suck At Machine Learning by dllthomas on Sunday November 20, 2016 @09:10PM (#13002546)
In a deterministic universe, there's a sense in which that's always the case. But really, probability is always "given the information available". "I know everything about a deterministic universe" is just a special case.

Related to the broader topic: https://en.wikipedia.org/wiki/Scoring_rule#Proper_scoring_ru...

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @07:57PM (#12991330)
The fake part comes into play if you think they lied and pushed the wrong results, but presented them as "scientific".

Note it was not just a small mistake. They presented Hillary having a massive landslide, and instead she delivered the worst Democratic performance in 20 years.

Something doesn't add up...

Re:Facebook Must Really Suck At Machine Learning by morgante on Friday November 18, 2016 @08:06PM (#12991364)
Frankly, this is an incredibly stupid argument and I'm sick of hearing it. Incorrectly forecasting the future is a far cry from incorrectly reporting facts. Nobody said it was a fact that Clinton would win.

Where do you think they "lied?" Even simple poll averages showed Hillary was ahead.

I'm willing to admit that they overestimated the chances of a Clinton victory. Nate Silver's 60% was a lot more realistic than the 90% the NYT was pushing. But having a poor prediction model is a lot different than falsely reporting the past.

More importantly, it's totally incoherent. If the MSM was tilting their model to try to get Hillary elected, the worst thing they could do would be to present it as a sure thing (depressing turnout).

Also, for the record, Clinton won the popular vote. So the polls got the popular vote percentage right.

State polls were a lot more noisy and infrequent—it's not that surprising that there was a polling miss.

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @08:31PM (#12991467)
> Frankly, this is an incredibly stupid argument and I'm sick of hearing it.

Sorry. Didn't know you heard this before. Yeah the argument is misguided perhaps, but "incredibly stupid"? I don't know, I think you're being a bit harsh.

> Where do you think they "lied?"

They lied who they polled and how they fudged the results

> Incorrectly forecasting the future is a far cry from incorrectly reporting facts.

It is not far cry. This is one organization which has lost its credibility because it routinely lied and misconstrued facts, cheated during debates. I don't see as suspicious that they fudged the models to make her look good as "incredibly stupid", but again you have your opinion, so that's cool.

> More importantly, it's totally incoherent.

Sometimes maliciousness and coherency doesn't go together. Again, I would say it might be "mildly incoherent" but not "totally". You can just as well say they wanted her to look good to discourage anyone voting for the other candidate, which indeed jives with the rest of the rhetoric.

> Also, for the record, Clinton won the popular vote. So the polls got the popular vote percentage right.

What does that have to do with anything? Are you saying, she is a top experienced politician, lawyer, with a stellar team of experts, media behind her back, President, DNC, may large corporate donors and she didn't know that she had to win electoral college votes and just got a little confused?

> State polls were a lot more noisy and infrequent—it's not that surprising that there was a polling miss.

Somehow the "idiot" Trump managed to poll in the right states at the right time make Hillary deliver the worst Democratic election results in more than 20 years. If a dummy can do it, surely it was possible to poll properly.

Re:Facebook Must Really Suck At Machine Learning by morgante on Friday November 18, 2016 @08:41PM (#12991509)
> They lied who they polled and how they fudged the results

Care to provide evidence that dozens of independent pollsters all colluded to fudge results?

> It is not far cry.

Yes it is. Do you really not see the difference between me saying "it's raining outside" (when it's not) and saying "I think it's going to rain tomorrow" (and it doesn't).

> What does that have to do with anything?

It has a lot to do with it. The primary input into most forecasting models was national polling data. And national polls ended up being correct: Clinton won the national popular vote.

I'm not sure why you think Trump managed to poll better than the media. He spent months railing about how the election was rigged and he might not accept the result—if he was convinced he'd win, he wouldn't be wasting effort on that.

You're not even mentioning betting markets. Do you think thousands of independent people bet money in order to incorrectly portray Clinton as having a lead?

If you want to assert a global conspiracy to subvert reality, please bring data.

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @09:17PM (#12991649)
> "I think it's going to rain tomorrow" (and it doesn't).

Not if your job is to predict rain and you somehow completely missed the that there's going to be a massive rain-storm. And you also happened to own a company which sells umbrellas on the street corner.

> It has a lot to do with it.

Well, ok, I take it back. It has a lot to do with them failing to predict it because they relied on the popular vote.

> Care to provide evidence that dozens of independent pollsters all colluded to fudge results?

I didn't say they colluded. They fudged their models by being in denial and wanting to see a certain candidate win. Someone doing it as a hobby is understandable. But when they are paid a pretty penny for it, then it borders into dishonesty.

All actors doing the same thing doesn't imply it is a cabal and they all colluding. They simply have to have the same incentives.

Surprisingly one of the polls that does measure enthusiasm the Daybreak poll from USC did predict a win for Trump http://cesrusc.org/election/

> Do you think thousands of independent people bet money in order to incorrectly portray Clinton as having a lead?

The probably took the cue from the media.

> He spent months railing about how the election was rigged and he might not accept the result—if he was convinced he'd win, he wouldn't be wasting effort on that.

But wasn't that a much better strategy, as overconfidence lead to people staying home. That idea is probably not novel to him or his team!

We've all heard how dumb he is, yet somehow he knew exactly which states to go to, doing up to 6 speeches a day, in seemingly just at the right places. He got lucky a few times, maybe in primaries, but he seems to keep getting lucky consistently.

> If you want to assert a global conspiracy to subvert reality, please bring data.

Would you say reality was subverted already? We have a TV personality as a President. This is the stuff they put in comic books. Conspiracy here is called overconfidence, cockiness and complacency. That's the conspiracy. We can make fun of and call Trump supporters names as much as we want. But in the end Hillary had a very expensive gun (the media) and she went ahead and shot herself in the foot with it. I think that is one of the primary reason people didn't vote -- they saw the map with 400+ electoral college votes in her favor and 1% chance for Trump to win. But in the end it is those pesky KGB agents and Fake News that we are blaming... The more we blame them. The more Trump will keep winning.

Re:Facebook Must Really Suck At Machine Learning by enraged_camel on Friday November 18, 2016 @08:06PM (#12991365)
Note that national polling was fine. It was the state polls that turned out to be hugely inaccurate. That doesn't mean they were "unscientific," but that they made some mistakes with their polling methods.
Re:Facebook Must Really Suck At Machine Learning by rebuilder on Friday November 18, 2016 @08:06PM (#12991367)
Perhaps the electoral system is the missing variable you're looking for.
Re:Facebook Must Really Suck At Machine Learning by dragonwriter on Friday November 18, 2016 @08:15PM (#12991416)
No mainstream prediction I saw predicted a massive Clinton landslide; most predicted a slightly larger popular vote victory than actually occurred.

Some, though not the best regarded, gave a Clinton victory a very high probability, largely because they modelled state-level variability as independent rather than highly correlated, an error that 538 called out explicitly.

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @08:35PM (#12991483)
Most were also biased in her favor. That was the big learning experience for me in this election. She had an unprecedented support from media, unlike few other candidates. I think even Fox (the traditional Republican faux-news channel) didn't like him very much.

Yeah I don't know the answer, what the mistake was. I'd want to know. I suspect some never responded or wanted to participate. Those who participated, said they'd vote for someone else but changed their mind. They used registered phone numbers and some people more consistently paid to have their number de-listed...

Re:Facebook Must Really Suck At Machine Learning by hga on Saturday November 19, 2016 @07:07AM (#12993114)
Yes, from what I've read Fox was very anti-Trump, aside from a few of its figures who's contracts have not inspired.

I mean, this election was used as an opportunity to mount a successful coup ousting its founder, who'd, BTW, announced a couple of years ago that the network was moving to the left anyway....

Re:Facebook Must Really Suck At Machine Learning by sndean on Friday November 18, 2016 @08:10PM (#12991385)
> instead she delivered the worst Democratic performance in 20 years.

Elections don't happen that often... so it's just a comparison against fellow losers Gore and Kerry.

And Kerry lost by both the popular vote and the electoral vote.

Re:Facebook Must Really Suck At Machine Learning by rdtsc on Friday November 18, 2016 @08:57PM (#12991565)
Bill Clinton and Obama, they had a better numbers. Dukakis only was worst and was early 90 or so. And that is despite her having such a well funded campaign.
Re:Facebook Must Really Suck At Machine Learning by hga on Saturday November 19, 2016 @07:12AM (#12993133)
And anyone who knew Dukakis and Flyover country, as I did having been raised in the latter and then moving to Massachusetts for college in 1979, knew it was very likely Dukakis was going to lose.

Oh, wow, here's a Wikipedia article worth looking at, "Historical polling for U.S. Presidential elections": https://en.wikipedia.org/wiki/Historical_polling_for_U.S._Pr...

Re:Facebook Must Really Suck At Machine Learning by mturmon on Friday November 18, 2016 @07:43PM (#12991252)
Well, there's one data point.