News for hackers. Stuff that matters.
Re:A Way to Reverse Coronary Artery Disease? [pdf] by brettp on Wednesday July 02, 2014 @12:32AM (#7975197)
1. Awful summary

0.6% recurrent event rate does not mean it was 99.4% effective. You need to know what the recurrent event rate would have been if everything else was the same and they hadn't followed the diet. Hint: if the normal recurrent event rate is 0.3% then they are worse off, not better (insert test of statistical significance here).

If the event you are testing for (recurrent CVA) is rare, then a large enough sample and proper statistical significance tests are even more important.

2. No control group

"Didn't follow the diet" is not a control group. They have, post-hoc, selected the subgroup consisting of "people unlikely to follow a lifestyle intervention". That group might also been the most likely to continue smoking, not take their medication, do no exercise, be too unwell to follow a rigorous diet, etc etc.

3. Subgroup analysis

Drawing a circle around a subgroup after your study and making inferences from them is an excellent way to get bogus results. (Classic example: Wimbledon winner prediction system. Email 2000 people at random with each player and ask them to bet $1 on that player. After you know the winner, get the subgroup of people who won, and ask them how accurate they think your winner prediction system is.)

There are no comments on this comment yet.