The issue is that dividing the participants after the fact and then looking for correlation in the existing data reduces the significance of the statistic considerably (we have other statistics for that). The p-value is not representative when used that way.
But if you do another study focused on that group in particular and still get a significant result, you're fine! The problem isn't that they located a group on which the drug worked in a dishonest way, or some such - the problem is that they were dishonest to claim they had significant evidence that the drug worked on that group. If they'd done an additional study on that group in particular, they would have their evidence (or, of course, a null result).
At that point, his product would look great, hiding its failures.
This way, while he misinterpreted the P value in an illegal and fraudulent way, he did release all relevant information - ironically, better for the informed reader than if he had rerun the trial legally.
It would be absolutely fine to run a new trial on the supposedly most beneficial population (those with mild/moderate lung damage; lets call them 'the subpopulation').
If that second trial succeeded, then it would be strong evidence that the drug was beneficial for the subpopulation.
There would be no need to hide the results of the first trial, as the first trial did not provide evidence that the drug didn't work on the subpopulation.
If you read the article to the end, they did in fact do such a trial on the subpopulation. And they got evidence it wasn't working on the subpopulation - which is how science goes.
The problem was that the first trial wasn't set up to examine the subpopulation, but they reported results as if it was. You can't do that with standard NHST, as it invalidates the assumptions of the statistical framework being used.
But you can absolutely decide to run a whole new test on a new sub population, based on hints you get from the first results.
And, while it'd in general be better if all test results (positive AND negative) were published, that is not relevant to this situation - the first trial said nothing bad about the effects on the subpopulation, so there'd be nothing to gain from hiding it, if you just wanted to claim it worked on the subpopulation.
Its not like a situation where they got evidence that the subpopulation would not benefit in the drug in the first test, and then decided to do another test, planning to only report the second.
However, if the results of the original test are hidden, the results of the second test could well be taken as evidence for a wider or stronger effect, yes? If this isn't the case, then I wouldn't see a problem with that behavior - but from the reading I've done, I suspect it is in fact the case and is common practice.
edit: I may be completely wrong on this - if, indeed, that's not a significant problem.
what i think you're saying is that they would hide the original negative study and publish a subsequent (new, separate, on different people) positive study.
[aside - that's not a perfect description because for one particular group the first study was positive; it's just that the group in question wasn't explicitly targetted].
and, in general, that's considered a bad thing. because (1) you can keep repeating studies until you get a positive, and then publish and (2) because the negatives aren't published, people have incomplete information.
but it's not a terribly bad thing, because if something isn't true then, if you repeat a study, it's likely going to show it isn't true. the standards are set high enough that you'd need to do hundreds of studies before you showed something to be true (when it really isn't).
and because hundreds of studies are expensive, it's unlikely to happen (but then you think of the industry as a whole, and it is doing hundreds, and so some of those are likely wrong...).
in contrast, what this guy was prosecuted for was hunting in the data. you can think of that like doing a new study, but without the cost. it's pretty easy to dream up hundreds of different questions you can ask existing data. and just by luck you're going to find the occasional surprising answer.
so hunting through data is like doing hundreds of studies until you find something, but it's cheap! and that's why it's "worse" than simply hiding negative results and repeating studies. because it's much more likely to happen in practice.
I don't thing that you're disagreeing with them, just reiterating.
This is how a lot of Science gets done.
It was significant enough to stop the trial.
(http://www.ncbi.nlm.nih.gov/pubmed/19570573?dopt=Abstract)
> FINDINGS: At the second interim analysis, the hazard ratio for mortality in patients on interferon gamma-1b showed absence of minimum benefit compared with placebo (1.15, 95% CI 0.77-1.71, p=0.497), and indicated that the study should be stopped. After a median duration of 64 weeks (IQR 41-84) on treatment, 80 (15%) patients on interferon gamma-1b and 35 (13%) on placebo had died. Almost all patients reported at least one adverse event, and more patients on interferon gamma-1b group had constitutional signs and symptoms (influenza-like illness, fatigue, fever, and chills) than did those on placebo. Occurrence of serious adverse events (eg, pneumonia, respiratory failure) was similar for both treatment groups. Treatment adherence was good and few patients discontinued treatment prematurely in either group.
"That was the death knell for the drug. Most insurers stopped paying for it."
I don't think they're implying that the drug killed people. I think they're saying that the study made it obvious it wasn't helping, so insurers stopped covering it and other consequences followed.
He could have just kept that data secret, and ran another trial but specifically targeted at people with mild to moderate illness. That would have protected him legally, and made the numbers look even better.
That's the kind of thing that many people are campaigning against. Companies should release all the research they do rather than cherry picking the useful (to them) results.