1. There is no control group. Since people get better with cognitive tests with practice, we don’t know whether these improvements were due to the treatment or just the normal practice effect. (Though if I were a betting person, my money is on the latter.)

2. Cognitive tests are not noiseless. The same person may vary by a few points between repeated tests. The modest improvements on most tests found here might not be above this noise floor.

3. The most impressive result (memory) is almost certainly regression to the mean. That is, the group did unusually poorly on this task in the first session by chance, then they performed more typically in the second, which looks falsely like a big improvement.

Basically, I think this paper provides no meaningful evidence of cognitive improvement in humans. I can’t speak about the mice stuff, that might be fine.

I'm a little surprised they wouldn't have done a larger secondary trial before publishing, the impact would have been much greater if the effect was still observed in a larger group, even if they still did not have a control group- or perhaps had an alternative treatment schedule (shorter exposure times- is treatment duration correlated with degree of improvement? Then you have less of a moral dilemma of denying participants of a useful treatment while still getting some data that would indicate the treatment itself actually does something).

(working on a PhD in genetics)

Resource constraints are a much more plausible explanation - running the study with a control group would cost twice as much (in time as well as money). This is either a pilot to apply for funding to run a better-controlled trial, or it's just not scientifically serious. Both are very plausible, in my experience.

I do feel confident in saying that Scientifically, the human data in this study are largely uninformative.

Should be called "collected case studies" or something.

Edit: One way to think about this is assuming the effects are random and symmetric then the probability of going up is 1/2. If all 6 participants had increased scores, then the p-value is at least 1/2^6 and this is not taking into account the magnitude of the shift.

https://en.wikipedia.org/wiki/Aging_(journal)

> The peer review process employed by the journal has been criticized by Jeffrey Beall, a university librarian and expert on predatory open access publishing, who also included the journal and its publisher on his list of "potential, possible, or probable predatory scholarly open-access journals" in July 2015.

106 is well within a standard deviation of both of those. In fact, 109.5 is only one standard deviation away from the first, and the reverse is almost true. How do they get p = 0.004?

If this is a significant treatment of Alzheimer's then that's a breakthrough to me, what bar would you set?

Wouldn't having some kind of proof against a placebo or at least a dose-dependent improvement be the least possible requirements to actually believe the paper found any kind of effect at all?

If you randomly give 6 Alzheimer's patients an apple a day for 5 weeks, they could show improvements in cognitive tests by pure accident, especially for a disease that is known to ebb and flow.