Many who are published deserve obscurity. And some who have obscurity deserve more funding. Can you give it to them?
Do not be so eager to deal out censorship; for even institutional review boards cannot see all ends.
Should there be consequences for people who publish buggy code on github? Because that situation is completely equivalent to what these phd students did. ArXiv isn't a repository of peer-reviewed accepted papers, it's just a repository of papers, just like how github is repository of code.
The people to blame here aren't the students themselves. They just wrote up a mildly interesting observation and put it up on a repository. The people to blame are journalists and internet dumbasses who used this paper to serve their own selfish ends.
> I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled.
There's no denying the existence of a lot of peer-reviewed junk science but sometimes there are perfectly good reasons why papers with faulty conclusions are accepted. The question addressed by the paper might be interesting, the experimental methodology might be interesting, the data sources used in the research might be interesting, or the editors/program committee might just want to highlight the lack of progress in an area that deserves more attention.
Publishing something that is completely fault-free isn't a desirable or even an achievable goal. This is a fundamental misunderstanding of the purpose of scientific publishing. The goal of scientific publication is to advance the field by stimulating discussion at the boundaries of human knowledge. This can often be done even with "buggy" results. Think of it as the scientific equivalent of fail often, fail early and fail fast mantra that drives many startups.
I'm not dismissing your frustration though because the problem you've described is a real one. We aren't very good at communicating scientific results to laypersons. But unlike you, I don't have a glib solution to a problem I don't fully understand. Magazines like the communications of the ACM with accessible, professionally-edited articles written by experts in the field are probably one step in the right direction, but there is a lot more that needs to be done here.
But much more importantly, the flaw in your mental model is thinking that claims in published papers are expected to be correct. They aren't. The goal of a paper is to advance the field by stimulating discussion and new research. The unfortunate consequence of this is that laypersons can't directly use research papers as an authoritative source describing our current understanding of a subject.
The github analogy still holds up. Non-programmers shouldn't be downloading code off github, compiling and running it because they have no way of evaluating whether the code really does what it claims to do. They're probably better off getting applications from curated app stores. Accessible scientific publications written by recognized experts in the field are the equivalent of curated app stores. That is where laypersons should be looking to understand the state of the art in scientific knowledge.
If we were to continue the GitHub analogy, it would be great if ArXiv had a public "issue" list.
In HN-speak, Arxiv is like a Svtle blog.
As a Princeton alum, I'm a bit saddened at the course of events :(
On the OTHER hand, this was a non-peer reviewed thought experiment by some MAE students outside their area. It would be akin to me turning in an assignment for a class and the media ran with it saying "Princeton study..."
Unfortunately, readers are going to read this rebuttal as an attack on Princeton, which it really isn't; if anything it's only funny because of how strong Princeton really is.
4500 undergrads, 1500 graduate students, and hundreds of faculty, and you think that every little paper they give or study they conduct is a product of the whole in a meaningful sense?
The study was practically just a thought experiment by two grad students looking at something outside of their expertise. It _only_ got any attention, because it was about Facebook, and so the media saw a chance to sell more ads. For the same reason, they used the metaphor of pars pro toto as a rhetorical ploy, because Princeton vs. Facebook sells more ads. Like many among the unthinking masses, you got punked by it.
They submitted their paper to a well known, well regarded preprint service (arXiv), which is pretty normal for a number of fields.
Then, presumably, the paper would go to peer review. Where one would hope the reviewers would point out a few glaring flaws (like that an SIR model without either the creation of new susceptibles or waning immunity must go to 0 and they are 'dooming themselves to success').
The paper would get rejected, or heavily revised, and the world would keep spinning.
They didn't actually do anything wrong. If you want to blame someone, blame an overexcited media that grabbed something off a preprint server and ran with it, knowing full well arXiv has no 'Is this right' quality checks, and isn't peer reviewed.
> The paper would get rejected, or heavily revised, and the world would keep spinning.
Which means they did something wrong. Sure it might have been just a mistake, not quite the cardinal sin the GP makes it out to be, but you don't have to publish papers with flawed arguments on ArXiv.
It's one thing to write up a silly LaTeX PDF with wild speculations, put it on your webpage, but you don't need to submit every brainfart to ArXiv either, or do you?
Beyond that, the paper isn't pants-on-head crazy. The objections most people have to it can be solved by adding a single term to one of the equations, and some very straightforward sensitivity analysis.
Which is, you know, exactly the kind of thing pointed out in peer review.
http://en.wikipedia.org/wiki/ArXiv
And it is standard for authors to list their university affiliations on papers whatever the quality of the paper regardless of whether it is a pre-print or not. Look at every other paper on arXiv, they all have affiliations, and there is a ton of crap on arXiv that will never pass pre-review.
These students didn't do anything wrong in putting the paper online here. Now if they did a PR to the media to promote their work and mis-portrayed it as peer reviewed or authoritative, that is a different matter. But putting a pre-print on arVix with proper affiliations is just standard procedure these days.
The question you pose here isn't whether or not something is "good science", it's about who gets to decide what constitutes "good science" and "bad science".
Science has been able to handle itself quite well over the centuries since it has come into existence. Publishing something, in your name, as these authors have done, is an implicit declaration of what you phrased as "I'm willing to stake my reputation on the idea this trend is real and will continue".
The spirit of free inquiry, on which science is based, demands that we hold nothing sacred and immune from questions. Good science, almost by definition, requires "bad science".
Be careful what you wish for, because you just might get it.
If you want to blame someone, blame an uncritical media.
Lab coats are there to bounde ideas off each other and sometimes write stupid things that will get corrected by other lab coats.
Consequences should be there for the people who find such articles and take them for a media spin. It's not the paper that is doing the damage, it's the news articles, like the Guardian story. It is the responsibility of a journalist to make sure they're not posting utter bullcrap to millions, and when they fail to do that, there, here should come consequences.
So what's been thrown in your face, the published paper or the out-of-breath news article with the click-bait headline?
Studies are only ever useful in aggregate, anyway. It's about metaanalysis, not just what any singular study says.
There should be consequences for the people who publish these things. People have a tendency to believe anything that someone in a lab coat says, especially if it supports their point of view or anecdotal experience. In many cases the people who do the research present it with few qualifications while not standing behind assumed implications. If someone publishes sensational and link baity findings they should say, unequivocally, "I'm willing to stake my reputation on the idea this trend is real and will continue" or "These are just data and I'm not willing to say that they have any bearing on reality".
Facebook may not have been right to dignify the initial post with a response, but I hope it works for the best. They say that some attention is better than no attention at all. It's important that this applies to self promotion and persona creation and not science. If somebody has something crazy to say, they should start a personal blog. Those who want to intentionally attract media attention should present themselves as such, instead of pretending to be doing any kind of meaningful experiment and hypothesis testing.