Sample size isn't as big of a deal-breaker as the layperson may think. If the mechanism of action is well understood and the potency of the effect is strong, you don't need a big sample size. For an extreme but apt example of this: how many people really need to be beheaded to prove that it kills someone to a reasonable degree?

Wow, motte and bailey.

Hengist et al publish a toy paper where they fed 14 guys pizza and ran a blood panel. Sure, why not. Probably took an afternoon, and was real cheap to do, gave some of his grad students publication credits and LaTeX experience. We learn that eating a pizza will not instantly give you diabetes, or make you obese, which we already knew. This, by itself, is fine.

Separately, the University of Bath has a press office which is required to issue a certain number of press releases per year, even when the research isn't interesting. Again, no real problem. Plenty of press releases are issued with nobody ever reading them. If the University of Bath wants to employ people to do nothing useful, that's their own business.

This press release is submitted to HN and gets a bunch of upvotes. Problem. People in the comments try to generalize the study. Big problem!

The paper had no followup, no dose schedule, tiny experimental groups... because it wasn't trying to solve obesity!

Nobody knows what's causing the obesity epidemic. There are many hypotheses, all with holes and strange contradictions. There have been many gigantic studies, all with inconclusive results.

There is no reliable way to lose weight.1 There are many diets that work for some people, all with holes and strange contradictions. There have been many gigantic studies, all with inconclusive results.

This is a nice little toy study. I don't doubt the data or the results. I just don't think it's useful. And studying obesity certainly isn't like running a randomized clinical trial on if beheading will kill a person!

---

1: Yes, if you lock someone in a cage and not give them food, they will lose weight. Obviously. Once you release them back into modern society, they will regain that weight.

>There is no reliable way to lose weight

Correct me if I'm wrong but that is false. It seems pretty well known and accepted that caloric intake below BMR will make you lose weight. Your BMR might fluctuate, but if you take that into account there shouldn't be a problem.

Going back into society and resuming your previous eating habits (before the cage) will ofc make you gain weight because your BMR will have lowered during your starvation.

Oh I love spotting motte and bailey fallacies. Didn't intend that here. The comment I replied to only pointed at the sample size.

How many other factors than beheading can affect a persons risk of dying from being beheaded?

A huge number. People die without being beheaded all the time.

The relevant statistic is not how many people die, it's how many people die in a short time window. The probability of a particular person dying in a 5 minute interval after being beheaded is much larger than the probability of a particular person dying in an interval after they were not beheaded.

That just shows that beheadings are targeted at people who were already about to die.

Yeah I think it really reveals a breakdown in our ability to quantitively model the intuitive notion of "causality".

A targeted public health intervention that prevented beheadings just at the time of beheading would clearly break the link in the causal change. This result is insensitive to any imaginable confounders. However, the executioner would find another strategy, say a gunshot or noose, which did not appear anywhere in our dataset.

Really makes you question the validity of observational studies. At least when acts of government are concerned.

Yes the answer is more than 1, but because the mechanism of action is so well understood and the temporal association so strong the answer is less than 10.

> If the mechanism of action is well understood

In OP's paper, I'm not sure if they met this criteria.

That's not a good example because we (implicitly) make conclusions based on the very large background control group of people who rarely die.

Over what time period?

Sure, if you measure the time surrounding the beheading, it's obvious, but if you expand the time range far enough, then you find out that everybody dies whether they're beheaded or not and suddenly it looks irrelevant.

Yup, worthless. It doesn’t account for different metabolisms, age and eating habits sufficiently to be considered worthwhile.