> Further research with a placebo-control arm is warranted
IIUC they don't have a control group. They don't have a control group!!! Reading again your comment, you wrote this in point (3) and (3b), but it needs a few more exclamation marks.
Again, parent poster mentioned this but as you say, more exclamation marks are needed.
More importantly, the peer-reviewers are likely scientists, not physicians. This can make a subtle difference in how and what they review.
But you did a great job listing out some of the shortcomings!
As for one of the senior authors filing a patent for commercialize probiotics for autism, shouldn't there be a mandatory disclosure in their papers that have a conflict of interest?!
In this article the patent applications are disclosed in the Competing Interests section. The readers can draw their own conclusions from those. It would be very unusual for a journal to not demand a declaration of conflicting interests.
Also I know SciRep is fairly new but isn't it still under the Nature umbrella? ie. There's still some peer review; the study is likely to be legitimate.
The authors are pretty explicit about the weaknesses in their study and I think that the flaws are reasonable for a pilot study. I'm surprised the authors didn't say anything about their conflict of interest though.
Showing a link between autism and the gut microbiome would be incredible. Obviously more work needs to be done before anything is concluded but this is a really interesting result.
The publication incentives are extremely messed up and in favor of exciting results that aren't true.
Thus, this paper is saying what the scientific method should test next.
The number of subjects was small. They even added: "We note that due to the open-label nature of this initial trial, all of the assessments are subject to placebo effect..."
Step 1: Observation
Step 2: Hypothesis
Step 3: Test
Also, what exactly is SciRep?
tl;dr: "In this other observational study, many subjects with autism didn't improve over time on a different metric, so the improvements in our metrics were probably because of the treatment and not spontaneous. Also, we asked the parents if there was any regression to the mean, and they said no."
there are remarkable parallels to Andrew Wakefield's paper about autism, vaccination, and his hypothesis about the gut.
Perhaps this should have been published in the Lancet[1]
https://www.thelancet.com/journals/lancet/article/PIIS014067...
(2) Nature is an extremely high impact factor journal. However, it's ... not really a medical journal. Nor a psych journal. Nor a developmental peds journal. It's a basic sciences journal. What I mean to say is, even if this were Nature, it'd be a super high-impact journal that isn't actually read by clinicians who would be best positioned to criticize the study.
(2a) For instance, their bowel transfer prep involved a clean-out with a two week course of vanco. The only bowel bugs it's used for are c. diff colitis and staph aureus enterocolitis. The reason it's really only used for that is because it really doesn't touch any other common bowel inhabitants. It's useful for those because it doesn't absorbed very effectively, so if you've got C Diff sitting in the bowel lumen, great, send a wash of vanco down the pipe. But, uh, for any other bugs in the gut? It's not even second line. So, how weird that it would be used to knock out pre-existing gut flora and "suppress pathogenic bacteria." C Diff and MRSA are not the leading pathogenic gut bacteria. The pre-existing flora are predominantly anaerobes.
(3) An unblinded, uncontrolled trial of a disease whose symptoms are highly subjective in measure? Excellent. I'm sure they took measures to ensure they weren't influenced in their subjective analys-
(3a) Oh. The senior author has a bunch of patents out trying to commercialize probiotics for autism.
(3b) Oh, they used the PGI3, CARS, SRS, and ABC scores. I guess if you fill out enough bubbles you can overcome the fact that those are still subjective enough that they're shit for an open-label trial. Whoops.
I don't mean to be overly dismissive. It's a fine pilot. The results are interesting enough that I'd want to see a real study tackle this. My comments are mostly aimed at the prevailing sentiment here taking this at face value.