Because journalism is a systematically dishonest profession populated largely by people who do not want their readers to be able to catch their mistakes and do not want to give credit to the researchers who made the discovery that earned the clicks for the news article. They care about entertaining, not informing; the truth or falsehood of what they are writing is irrelevant to them, except as a constraint that might impede the telling of a sensational story (all else being equal, a story is more sensational if it is true than if it is fiction). Like PageRank-hungry SEO scum, they will only name sources if they think some of that the source's credibility will rub off on their article, never vice versa.

It's really depressing how cynical that is. (Developer working at the NYTimes here).

The actual reason links to studies are often left off articles (particularly in cases like this with sharp deadline pressure to publish fast before the competition) is that the software powering the editing and publishing workflow really badly needs improvement. An incredible amount of work and knowledge goes into a story like this.

Versioning rich text through many different software tools designed for writing/editing and publishing across many platforms is hard. Sometimes people copy/paste by hand and in doing so a link can go missing. The news industry needs more technologists to work on these problems. We're hiring for people to do that, by the way: http://developers.nytimes.com/careers/

(Also, the link to the study is now in.)

It's cynical because that's what we, regular readers of news, have to deal with. NYT may be top of the line, but sadly it's not above the line.

Now I don't want it to sound dismissive or personal in any way, but tell me - if say, few of great software devs now drop everything they do and come to NYT to help, sit down for months and develop the most awesome software package the world of press has ever seen, will it actually solve the quality issues articles have? And more importantly, if sold to other papers, will it suddenly solve their problems?

Will it make journalism honest and trustworthy instead of lies and clickbait bullshit?

I'm not sure how much blame to put on broken publishing workflow, a lot of this seems really to be about broken incentives - "deadline pressure to publish fast before the competition" that leads to the "many articles, as sensationalist as possible, truth be damned" mentality, especially in the management layer.

But you did give me a pause here. Only recently I had a chance to peek at internals of a tiny part of manufacturing industry, and oh boy how much money they waste on badly designed software, which is badly designed because of deadline pressure and top management pressuring to iterate over a broken software package (and then messing with the process) instead of scrapping it altogether and doing it right. Maybe software is more to blame than I thought.

> Now I don't want it to sound dismissive or personal in any way, but tell me - if say, few of great software devs now drop everything they do and come to NYT to help, sit down for months and develop the most awesome software package the world of press has ever seen, will it actually solve the quality issues articles have?

This actually happened and that's how we got d3.js. It didn't fix journalism though.

Oh, didn't know the origin story; I only thought they just like it over there. D3 is absolutely awesome!

>Will it make journalism honest and trustworthy instead of lies and clickbait bullshit?

You get what you pay for, which online tends toward nothing.

It turns out that paying people to talk to you, although it can work, tends to get them to tell you what you want to hear. Often, this is "lies and clickbait bullshit." (Think of a stereotypical Soviet government report.) Markets work very well for lots of things, but they can't establish honesty and trustworthiness. Instead, they need honesty and trustworthiness to function.

Still, it's better than them getting paid to tell you what someone else wants you to hear.

I don't recall ever having a problem with any actual reporting.

My objection is with the editorials. I've all but stopped reading "the news" (NYT, WSJ, Economist, my local paper, etc) as a result.

This is the best sort of thing that happens at HN.

"I don't like [thing]" "I work at [thing], come help me fix it"

You miss that the subject changed from media in general to NY Times. It is like mixing apples and ... well, rotten pears.

(Not claiming that NY Times is perfect. Of course. There are other good sources, too. But NYTimes isn't like the media I grew up with. At 20 I realized that all their coverage of subjects which I knew beyond the surface was garbage, at best. My specific example is DN, the largest Swedish morning paper, but could be most of the media.)

I'd be happy to be corrected of course.

I'm glad to see that the link to the study is now in. Also, the current version of the article includes phrases like "the study’s lead author, Dr. Robert Lustig"; did it originally? I can't find the link to the version history of the post.

I'm skeptical of this "software" explanation. Software can of course make citation management much easier, but I see lots of articles that don't even bother to mention the lead author of a cited publication; and how did the software get that way in the first place? The software reflects the priorities of the corrupt organization that produced it.

Having a chance to peek at a tiny area in manufacturing process, I can sort of imagine software replacing a more manual process by something faster, but much more messy and generally worse. It starts with three layers of corporate management above a subcontractor hired to write the software. Of course all requirements go through the entire chain, in what resembles and adult version of the game of telephone (aka. "deaf phone" or "Chinese whispers", the latter being particularly appropriate since what I saw, I saw in China...).

Somebody could probably sit down with the journalists for few weeks and come up with a software package that would fit their needs perfectly - if the development consisted of direct communication between the developers and journalists/editorial staff, without any management middlemen in between. Alas, that's not how software is procured in large organizations.

But I'm also skeptical. It could explain NYT's problems, but it doesn't explain the even worse problems every other news outlet has.

The reason articles about studies leave out a link to the paper is that the software doesn't support it? Why not just put the URL in, or just cite it normally?

Yeah. I can imagine they have lots of problems with preventing tidbits of information from getting lost in an environment of multiple people chaning the same block of free-form text, but if the article is literally about some particular paper, then starting the file with:

NOTE: Study "Effects of X on Y given XYZ" by Researcher N. Here, http://address-to-paper.org

(not the placeholder, the actual data) should help. I don't see such a line getting lost, and even if they happen to publish it by accident instead of incorporating into the text (as sometimes happens, things slip through), it would still reach the same goal anyway.

Good luck in your work, but unfortunately it wasn't the tech stack that prevented NYT from publishing the warrantless wiretapping story in due time.

The NYT managed to put together an absolutely massive printing and diatribution operation. If editorial cared about putting a citation in a story, they would do it. Getting a journal citation right is no harder than spelling the scientist's name right. Tech talent is not the blocker.

> sharp deadline pressure to publish fast before the competition

Sounds like the problem lies here and not on the technical side of writing, editing and publishing text.

I wonder what part of this decadence is imputable to ad revenu. If I were very naive, I would say all of it.

What a refreshingly cynical argument!

Cynical, yes. Accurate, unfortunately so. I worked for a small weekly paper for a number of years. Weeklies are still very community oriented and immune to many of the diseases that afflict large city dailies or national news outlets. Still, if you aren't selling advertising, you aren't in business, and advertisers don't want to see their ad next to something that either offends sensibilities or bores the hell out of readers.

You can wish it weren't so, or call it cynical, but that IS the reason they don't add a link to the source in the article. God help us all if people clicked away from the site, learned more about the topic than the reporter, or had the piece go over their head.

(Edit: I misread your "unfortunately so" as "unfortunately no" and I was super confused.)

I find your comment disheartening because I was kind of hoping that I was guessing wrong. Can I still hope that things are not quite as bad at the NY Times as at local weeklies?

It's just a guess.

My local newspaper's website never links to outside pages, always to previous articles within the site. I assume it is to prevent readers to go wander off.

I was going to link to a piece Dave Winer wrote on scripting news some years ago:

http://www.scripting.com/2005/12/12.html#howToMakeMoneyOnThe...

Where he talks about encouraging outgoing traffic.

But one of the more interesting links in that post has gone "evil-dark" (expired, taken by squatters, endless unclosable javascript "your iOS app is inf1ected!!1one! Call us to fix it! Have your credit card ready!"

So by your newspaper linking only to itself, it could be that they actually hope to avoid this dead link behavior. (Although it would be better that it was a self link that went to a vetted link repository.)

I'm really not a web guy, so I'm surprised that this problem hasn't been solved for most content management systems.

Linkrot is a huge problem, and we need IPFS or a similar system to prevent it.

It makes it harder for the reader to find mistakes and harder for the paper to sensationalize. It diminishes the ability to use a study as either commercial or political propaganda (which is not to say the studies themselves aren't commercial or political propaganda in the first place).

Media outlets want viewership more than anything else and the way to get and keep viewership is to tell people what they want to hear. It's why media outlets always have a consistent "slant"; MSNBC people watch MSNBC because it aligns more directly with their worldview than Fox News, and vice-versa. There is minimal tolerance within the audience for a host or pundit that appears to disagree on important factors.

You'll never see Bill 'O Reilly and Rachel Maddow on the same network, because that network would not have any viewers; conservatives want their conservative beliefs reinforced, liberals want their liberal beliefs reinforced, and both will only consume media that performs that function. Most people are not open to having their beliefs challenged even slightly.

All that said, I do find such articles often do state the university and/or the professors that were involved in the study or studies. Some of it may just be convention established before hyperlinks existed, though in many cases, the actual study is not publicly accessible.

Joe Scarborough has a 95% lifetime rating from the American conservative Union, and he's on msnbc. Your statements are a lot more universal than the evidence supports and there are plenty of people who don't run for the hills if their network of choice has a diversity of viewpoints.

Personal POV doesn't have to align with what you spend your day spouting on TV.

"Morning Joe" is a different type of program than the primetime "rage pundits" and Scarborough is one of MSNBC's more moderate hosts, but he still spends his day pandering to MSNBC's core audience. Bias goes much deeper than the words that do or don't come out of a host's mouth; the networks sets the agenda, frames the debate, and procures the guests that will cater to their primary viewership target, and Scarborough passively rides the gravy train. Anchors are performers more than anything, putting on the show that the network has paid them to put on.

What politician is going to turn down the opportunity to have his face in front of tens of thousands of viewers for 2 hours every day, even if it means he has to play nice with the other side?

If there were plenty of people who wanted a "diversity of viewpoints", you'd see this in the marketplace. Could you point out one place where this is actually true? Even NPR, which has a much smaller profit motive and thus should be less concerned with viewership and more concerned with integrity, panders constantly to its audience and refrains from creating a "diversity of viewpoints".

How often do you hear a fair story on NPR about what the conservatives perceive as problems with abortion or same-sex marriage (meaning, a story that doesn't reduce these, either the beliefs and or the believers, to a gross caricature)? Never, because it would make NPR's listener base really mad, because they don't agree that there are problems with those things.

Very few, if any, news outlets are neutral. They pander to their audience's beliefs and they set the dialogue by choosing the stories to give airtime and feed to the base.

Even the "neutral stories" are framed to push a specific viewpoint. They usually go like this: open up with a brief, slanted statement of events. Call someone who supports your POV and ask for their comment. Spend 2 minutes making their argument and bringing it in. About 75% through, put on a 1 or 2 sentence clip from the other side that basically amounts to "we disagree because bad reason x" (bad reason provided either by editing the clip or selecting a bad rep of opposite viewpoint), and then follow it up with another comment from the first person, the person whose argument and authority you spent the first 74% of the story establishing, that says "Well, bad reason x is just ridiculous". Then sign off.

I guarantee you 90% of TV and radio news stories that discuss a news event in a supposedly "neutral" way approximate that pattern. They do it because they're trying to reinforce the beliefs that they believe will make them more money.

Because news organizations are still operating under a newsprint paradigm where citations are considered a waste of ink. Like I'm not saying they actually still think about column inches, but that's how this became standard and people are still just following the standard. This is actually a good litmus test to see if a news organization has moved to the online-first mindset or not.

It might be correlated to journals that charge $70.00 per paper. If the NYT publishes a story citing a journal, the journal may go back, see that the NYT never actually bought the paper and a journalist just got a copy from somewhere, and then try to sue the NYT. The 'study' may be a weasel word that is used to avoid this outcome. And yes, over one article this is frivolous, but if the journal waits a few years and amasses 100 instances where this happened, then the NYT may actually go to court.

That's definitely not the case... Journals provide news outlets with complimentary copies of papers specifically so that they can be written about. Often, but not always, they're even provided in advance of official publication.

Ahhh, so it is laziness then. I should have known, never assume conspiracy when idiocy will do.

How so?

Borrowing material from the library is a "thing" as far as I know.

I don't think you have to own a copy of something to write about it. That would be insane.

Because they are now competing with blogs for readership.

In my experience blogs are usually much better about citing studies than newspapers.

Most readers are not on university networks and so would not be able to read the study anyway, except in rare cases of open access journals.

So what? Whether or not you give someone credit for their work or provide evidence for a claim you make is not contingent on your audience's having immediate access to the fulltext of your source. If the source exists, you cite it, and link to it -- period.

It's not about the audience not having access.

It's that the reporter is getting access without having ever paid for it. (Pirating, reading from another article, etc)

Nobody could ever prove how the author accessed it. It just isn't even an issue, legally or otherwise.

Setting aside the fact that it is a non-issue, in any case many news agencies have various DB subscriptions, most public libraries have DB access, most universities have 'reader' accounts which grant access to these DBs for like $100 a year, etc.

Any reporter who wants to access an article legitimately can easily do so. Else, they really do not belong in the profession of journalism, which relies on practitioners being resourceful and researching things in order to convey accurate information to others.

Shoot. I could have sworn this thread was nested under Balgair's comment about journals that charge $70 per paper. I must have misread something.

I almost think that's a good thing. It would be a talking point for gathering support for widespread open access policy.

So only cite open access articles (like this one). It'd certainly be a win for the public.

The New York Times is especially guilty of this, the more so in its health and science reporting. I noted and specifically called out, IIRC, the LA Times for specifically referencing and linking a source.

The Times needs to get with the times.

I noticed nobody in the thread seems to be pointing this out, so I'll do it:

Scientific research articles are very often behind a paywall. So citing them wouldn't really give the reader much more information.

Because most news organizations are not rigorous and care more about clicks than journalism.