News for hackers. Stuff that matters.
Re:A mysterious company’s Covid-19 papers in top medical jou... by eric_b on Wednesday June 03, 2020 @10:49AM (#23403930)
> This is a responsibility for which journal reviewers can only ever take partial responsibility for both practical and intrinsic reasons.

So what you're saying is we should never fully trust these journals? We should always be skeptical?

That's what I'm saying too...

Re:A mysterious company’s Covid-19 papers in top medical jou... by PiggySpeed on Wednesday June 03, 2020 @11:49AM (#23404748)
If you're healthcare, you're taught not to trust results because it was published under a brand-name journal.

YES, you should be skeptical, but mostly not at the journal-level. You need to be skeptical at the article-level. That is why it's so important to be actually TRAINED to interpret the studies.

The abstract of a trial is like an "advertisement" for the study. You quickly scan it to see if the study is worth reading. If it is, you make multiple passes of the article, identifying biases, understanding the study context, calculating ratios and numbers, reading through the lens of your own practice, and a bunch of other things.

Re:A mysterious company’s Covid-19 papers in top medical jou... by rscho on Wednesday June 03, 2020 @12:37PM (#23405393)
Healthcare pros are minimally trained in research paper interpretation and are almost all unable to perform the most basic statistical or critical review work.

So yes, I'd argue that brand names are a big problem. You just have to see how proud people are when they are accepted in one of the major publication venues, and the prestige that results.

I think you are not being objective.

Re:A mysterious company’s Covid-19 papers in top medical jou... by cm2187 on Wednesday June 03, 2020 @03:03PM (#23407321)
Except that it on the sole basis of this publication that the WHO and many other health organisations suspended ongoing trials of this molecule. So let's not pretend they would have done the same based on some random obscure journal.
Re:A mysterious company’s Covid-19 papers in top medical jou... by panabee on Wednesday June 03, 2020 @12:24PM (#23405217)
i am in the process of compiling a list of ways to quickly spot flawed studies. do you mind sharing your best tips?
Re:A mysterious company’s Covid-19 papers in top medical jou... by raphlinus on Wednesday June 03, 2020 @12:44PM (#23405495)
There is a course on this (taught Autumn 2019 at U Wash) that will also soon be published as a book:

https://callingbullshit.org/

Re:A mysterious company’s Covid-19 papers in top medical jou... by lbeltrame on Wednesday June 03, 2020 @01:04PM (#23405763)
Andrew Gelman put it very well:

"No, the real scandal is that the respected medical journal Lancet aids and abets in poor research practices by serving as a kind of shield for the authors of a questionable paper, by acting as if secret pre-publication review has more validity than open post-publication review." [1]

[1] https://statmodeling.stat.columbia.edu/2020/06/01/this-ones-...

Re:A mysterious company’s Covid-19 papers in top medical jou... by macintux on Wednesday June 03, 2020 @10:50AM (#23403953)
No, you said you wouldn’t trust it at all and that it was 100% political.

HN isn’t the ideal venue for hyperbole.

Re:A mysterious company’s Covid-19 papers in top medical jou... by seesawtron on Wednesday June 03, 2020 @10:55AM (#23404021)
But you argued as if you were always trustworthy of these Journals before but from now on, you plan to be skeptical. Your future intentions are similar but for the wrong reasons.

One should always be skeptical of research and have dicsussions on the reported data and to what extent and in what context it makes sense. The media outlets summarizing the research papers very often fail to do that so as to skip the details and keep the readers happy because they assume "readers just want to know the superficial abstract knowledge, they can't possibly be interested in the critical analysis of the figures and the numbers so let's exclude that".

Re:A mysterious company’s Covid-19 papers in top medical jou... by eric_b on Wednesday June 03, 2020 @10:58AM (#23404060)
But in this case the media is acting very dishonestly. I remember every major outlet posting articles about how now HCQ was debunked by this study. I have yet to see a single retraction of those articles or any new articles mentioning this. Only the science publications are covering this part of it so far.
Re:A mysterious company’s Covid-19 papers in top medical jou... by JadeNB on Wednesday June 03, 2020 @01:08PM (#23405815)
But why should the actions of the media change your perspective on the trustworthiness of the journals?
Re:A mysterious company’s Covid-19 papers in top medical jou... by JoeAltmaier on Wednesday June 03, 2020 @12:05PM (#23404981)
Journals are, after all, a filter for high-correlation results using small data sets. By definition that includes every coincidental result out there. You can probably create a distribution of accuracy with enough historical data.
Re:A mysterious company’s Covid-19 papers in top medical jou... by ardy42 on Wednesday June 03, 2020 @12:59PM (#23405693)
> So what you're saying is we should never fully trust these journals? We should always be skeptical?

I think that exactly how good scientists proceed. This isn't my field, but my understanding is that research groups don't "fully trust" peer reviewed research, so they usually attempt to replicate the key results they plan to cite in their new research.

Re:A mysterious company’s Covid-19 papers in top medical jou... by macinjosh on Wednesday June 03, 2020 @01:25PM (#23406029)
I agree, but if you do this you are usually labeled a ‘denier’ of some sort in regards to the topic at hand.
Re:A mysterious company’s Covid-19 papers in top medical jou... by DiogenesKynikos on Wednesday June 03, 2020 @12:01PM (#23404924)
No one in science places 100% blind trust in any paper, even if it is published in a prestigious journal.

In fact, scientists often say, somewhat tongue-in-cheek, "It's published in Nature, so it's probably wrong" (the more revolutionary a result, the more likely it is to be wrong, but also the more likely it is to get into one of the top journals).

Re:A mysterious company’s Covid-19 papers in top medical jou... by natechols on Wednesday June 03, 2020 @01:03PM (#23405746)
Repeating what I've said in the past, anyone that has worked in scientific research for at least five years probably has a list of high-profile papers in their field that never should have made it past peer review. There's definitely at least one Nature paper in my personal hall of fame, and I'm still pissed off about it after six years.