In the more well researched followup by the same person, the author of Astral Codex, we see that the FDA did not block or slow the approval. It was that the manufacturer had no interest in even submitting a application. In fact, the FDA regularly granted usage exemptions, granted a broad exemption for research, and even directly funded one of the first trials even though they are not primarily a funding body.

The actual moral of the story maybe is that we should let third partys submit applications? It seems like there would be little risk of bad outcomes as they still need to do the approval process. It aligns incentives better as the people ultimately at risk can deploy resources comparable to their QoL improvement instead of the expected increased sales volume at current prices. There is a obvious problem that the production process is inextricably linked to the final output so a non-manufacturer has limited control and insight into a core component that introduces the possibility of regression, but it seems relatively doable to develop protocols for handling that if they do not already exist.

edit: To be fair, once the application was submitted it still took 6 years for approval. There are likely large procedural costs in doing the exemptions and testing that make inaction preferable. So, in some sense exemptions and lacking interest in approval may be problems of their own creation, but on net the overall outcome seems fairly reasonable. Actual improvements in procedural matters are likely too nuanced for a random layperson like myself to comment on.

The application isn't a single page form, it's hundreds to thousands of pages detailing everything about the clinical trials ranging from the success criteria for each phase, the doctors and hospitals involved in the trials, and even tiny details like how much blood is going to be drawn, when it will be drawn, and how it's going to be tested.

They're impossible to write without the participation of almost everyone involved in the process since a significant part of premarketing approval is quality control of the final drug manufacturing process.

I also did point out that the integral nature of the manufacturing process is a obvious potential problem. It, however, does not seem like a insurmountable problem. The specifications on sourced components and the quality control processes applied to them is already integral to the drug manufacturing process. This would just move it one level further up where the final drug produced by some existing production process is a “component”. They also already approve drugs for different uses and evaluate existing drugs for new uses, so it does not seem that qualitatively different from some existing approval procedures. I am not sure of the exact details and am not proposing a specific solution, but it certainly does not seem to be a deal breaker.

I don't think you understand premarketing approval (PMA). Companies don't "run clinical trials and then submit for approval". The order is reversed.

Companies submit a PMA application first that details everything about the process upfront, after spending a significant amount of time going back and forth with FDA scientists and advisors. Once the FDA officially accepts the PMA application, then and only then can clinical trials begin.

Modifications to the PMA application after it's accepted have to be negotiated with the FDA and inevitably lead to delays. Changes to the manufacturing and quality control parts of the application are actually some of the easiest to make as companies frequently make improvements, but the "minimum viable product" for the PMA application is a very high bar to clear. They have to be able to prove that they can manufacture the drug reliably long before they give them to patients.

Many years later, when phase III results come in they are evaluated against the success criteria in the (at this point many years old) PMA application. If the drugs works, the FDA approves the PMA application, giving the applicant the right to market the product only as detailed in the application.

> I also did point out that the integral nature of the manufacturing process is a obvious potential problem. It, however, does not seem like a insurmountable problem.

The problem isn't manufacturing - that's really the easiest part of the whole process. It's the intellectual property and the nature of the medical approval process. It's an upfront all-or-nothing investment of tens of millions of dollars, minimum.

There are no shortcuts and without the rights to the patent, it is for all intents and purposes insurmountable. When a company does have rights to the patent, though, it's no longer really "third party."

There's a lot of copy pasting followed by customization - often by "contract research organization" consultants who specialize in different types of clinical trials - but there are no "frameworks" as we would understand it in software engineering.

In any case, I agree with the sentiment here. People should have the right to choose.

They don't, though. They need a control group.

Remember that if 10 treatments are being tried at once, you can divide the pool of participants so that only a smallish percentage are part of the control group.

Now, there are any number of reasons someone might be rejected from the trial. If that happens, the company isn't going to just freely give them the drug anyway, because that wouldn't provide scientifically valid data. Also, no one would enroll in a trial (which carries the potential of getting the placebo) if they could get the treatment outside the trial.

https://astralcodexten.substack.com/p/book-review-from-overs...

> If your lungs can’t breathe well, doctors can put you on a ventilator, which forces air in and out. It’s hard to get ventilators working right. Sometimes they push in too much air and injure your lungs. Sometimes they push in too little air and you suffocate. There are big fights about what settings to run ventilators on for which patients. For a while, doctors fought over whether to set ventilators on high vs. low, with many experts in each camp. Researchers formed a consortium called PETAL to study this.

Then, an institutional review board blocked the study:

> Its director, ethicist Jerry Menikoff, had decided maybe it was unethical to do RCTs on ventilator settings at all. He asked whether they might be able to give every patient the right setting while still doing the study. The study team tried to explain to him that they didn’t know which was the right setting, that was why they had to do the study. He wouldn’t budge.

Yes, putting someone on the wrong ventilator setting could kill them! That's obviously evil! Unfortunately, nobody knew which setting was the wrong one! The only way to find out was to try both, in a systematic and scientifically-valid fashion.

The FDA thinks similarly. In late 2021, they canceled trials of Paxlovid because it was immoral to have a control group. They did not then _waive the trial requirement_, meaning Paxlovid was both too proven to deny but too unproven to be administered.

I understand control groups in general, those are fine. But this is specifically in trials where people are pinning their last hopes out of desperation on some medicine and some random drop of the dice seals their fate.

I'd imagine that somebody who has, say, a 10% chance to die in 3 month, is also quite desperate.

(b) most therapies don't improve prognosis in terminal cases and some make it worse. We don't know which beforehand. In the cases where the therapy makes it worse, was it therefore immoral to not put more people in the control group?

I'd argue the lack of knowledge makes it a wash morally. If we knew beforehand whether the therapy would be effective, we wouldn't need the control group, but we also wouldn't need the study.

It is rather akin to saying that I can't tell whether seatbelts reduce car-crash fatalities simply by comparing collision fatalities from 1967 seatbelt-lacking car models with otherwise-identical 1968 seatbelt-equipped models; I would need to actually randomly and secretly sabotage a certain percentage of the seatbelts from 1968-model cars, while deliberately ignoring everything I know about prior fatality statistics.

In medicine, you get confounding factors like change in standard of care year-to-year and facility-to-facility. You almost never get the kind of "clean-room" trials you get in mechanical engineering (and, indeed, the processes to create them would often themselves be unethical). So the best you usually get is trying to reason around (to torture the analogy a bit) "Well we introduced seatbelts three years ago and relative to the year prior to that, fatalities went way down... Oh wait, but COVID also happened the same year we introduced seatbelts and everyone stopped driving" if you compare a current trial to past non-trial standard care as the control group.

What is the Bayesian prior on "After receiving the COVID vaccination, patient died when they were struck by lightning?"

This is what I mean when I say the 'clean room' practices that are standard for most other fields of science would be considered unethical for medical trials. You can't, generally, lock people in a box for months to eliminate other confounding factors when trialing medicine; the best approach we have, therefore, is to minimize confounding factors by minimizing variables in space, time, &c. Yes, if we could lab-rat humans, keep them in controlled environments perpetually, we could use the previous in-a-box state as a comparison to the current state, probably. That's obviously a non-starter.

(There are some categories of medical research where time-variance has been used; it's just not generally an acceptable signal source, IIUC, for drug and therapy trials. If for no other reason than not nearly enough information has been captured about unrelated past patients to be compared to a current trial because relevant information to monitor was not known when monitoring past patients).

Mathematically not difficult to deal with, which is why you intuitively know the answer already, because it's approximately the same way your brain works naturally. The chance of dying when you are struck by lightning is about 10%,[1] which completely overwhelms the chance of the vaccine coincidentally killing you at the same moment by several orders of magnitude; while you could use the this datapoint if you want, Bayesian analysis will still confirm that it has approximately zero evidence supporting (or refuting) that the vaccine causes death, so Bayes' theorem effectively drops that patient out of the analysis. Which is the same thing you'd probably have done intuitively, unless some kind of formal controlled-test study rule prevented you from doing what would normally be the correct procedure. (Or worse, forced you to subject an unvaccinated control-group patient to a lightning strike, just to maintain control...?)

But, since your patient died from the lightning strike rather than surviving it, there is a small signal present that points towards "the vaccine increased the chance of the lightning strike being fatal", which is a postulate that deserves a high initial skepticism until someone can propose a mechanism. But if you notice that multiple people die from lightning strikes after getting vaccinated, such that their lightning-strike survival rate is below the unvaccinated population's, you can start to update on that, and if it keeps happening, it becomes worth your time to investigate a potential mechanistic explanation. Like, the vaccination causing people to be less sweaty somehow.

[1] https://www.britannica.com/one-good-fact/what-are-the-odds-o...

Sweatiness is a good example of the sort of thing that would be hard to determine by comparing to past sample groups, since it's a rarely recorded piece of data.

This mathematical procedure is usually called "applied common sense", and if you are attempting to argue that it's impossible, you would in the process be proving that nobody can ever learn anything about anything.

That is a valid concern and is one of the three known serious adverse events (SAE) experienced during the Moderna covid-19 vaccine testing.

https://www.fda.gov/media/144434/download

> As of December 6, 2020, there were 3 SAEs reported in the vaccine group: a 65-year-old participant with community acquired pneumonia 25 days after vaccination, a 72-year-old participant with arrhythmia after being struck by lightning 28 days after vaccination, and an 87- year-old participant with worsening of chronic bradycardia 45 days after vaccination. On FDA review of the narratives, none of these SAEs are assessed as related. There were no cases of severe COVID-19 reported in the study.

(/s if needed)

If you're interested in learning more, I would recommend learning about Judea Pearl's do-calculus, specifically the backdoor criterion. Although it isn't the end-all of causal analysis, it is a very useful non-experimental approach that will help you develop an appreciation for the benefits of controlled experiments.

As far as seat belts go, that's actually been something discussed a bit in economics. The main concern is if adding seat belts changes how people drive (confounding the actual effects of seat belts). The solution was to measure the protective effect of seat belts on occupants of the passenger seat in collisions involving drunk drivers.

The causality is pretty clear and while you might not claim that it double the life expectancy it would IMHO be immoral not to fast track the approval of that drug.

Given a treatment with a large effect size like penicillin, a causal link will be very easy to establish even with a suboptimal control group.

This is false. Control groups, preferably in double-blind studies, are an essential part of best practices. However, the bar for what constitutes evidence is much lower than that.

Evidence is any fact that makes a relevant conclusion more or less likely. Evidence need not be conclusive in and of itself. Over time, the cumulative effects of less-strong evidence can form the basis for a reasonable inductive conclusion. Sorry to be trite, but as an example, a person can have evidence that their romantic partner loves them without the use of an identical twin.

Insisting on gold standard medical evidence before allowing someone to undergo life-saving treatment—when the preponderance of the evidence that does exist supports that treatment—is, to be blunt, the very definition of depraved heart murder. It demonstrates a wanton disregard toward human life in the face of the known likelihood that it will result in some number of deaths.

More broadly, however, it is an extremely poor policy, in that this meme both weakens society's critical thinking generally and—among those who see through the elitist charade—significantly weakens the respect of those scientists who make the claim.

I wouldn't go that far. I think that a paradigm built upon a broader evidence base -- e.g. with far more extensive postmarketing surveillance of drugs -- would be more ethical, of more benefit to society, and ultimately more effective.

Double-blinded studies can miss, can be confounded by placebo effects, can easily be confounded by crooked researchers, and are frequently just plain unethical from first principles. I don't think that they should be considered "essential."

RCTs are the gold standard for demonstrating efficacy precisely because of the controls. I agree that they're not going to give you enough info about potential side-effects - a thorough monitoring program is definitely necessary for that.

But without RCTs, showing that a drug works would be much harder. How would you weed out the snake oil?

> How would you weed out the snake oil?

The current paradigm is "better 10,000 people die of neglect than 1 person die of snake oil or quackery." I'm okay with a little bit of snake oil if it means that more drugs are being introduced more quickly, and I trust practitioners and patients to, more often than not, determine the therapeutic regimens that are right for them.

Compare that to establishing a bespoke chemical process for each drug, or messing with purification of natural biologicals, or fusing antibodies to protein or peptides to grow inside a whole cell.

They should be actually cheap. They aren't because new.

Humira is quite a bit harder to manufacture cleanly than a peptide or mRNA, and still harder than a plain solid state synthesized and folded protein. It requires a lot of post translational modification, which means cellular machinery, which means a whole bacterial cell you now have to grow, feed and remove from the product.

Agreed that there should be a faster way to trial drugs and get them to market. Though I've heard that the "slowness" of the FDA is actually overstated; not sure how true that claim is.

Any statistically significant difference there can then incentivize a drug research company to invest in the (expensive) clinical controlled trial process.

It's strange that people here are actually saying it's better to waste perfectly good research opportunities and watch patients die, than to risk having anything less than perfect evidence of efficacy.

The company would still need a blinded control group to be able to demonstrate the effectiveness of any treatment.

However patients should be allowed to enroll for free into multiple treatment trials at the same time. For a given number of people and treatments, enrolling in multiple trials simultaneously gives more science output, and more chance of success for the participants, if you assume that ineffective treatments rarely are very harmful.

But this would likely mess up any data, you can get. How can you tie the cure or the bad sideeffects to any drug then?

Apart from that, sure, I think patients should have the right to try anything.

Also: I can imagine that most of the patients would be eager to engage in all possible treatments. You might not even have a control group that did not participate in the study of the faulty drug X

That's not at all how clinical trials work. If and when someone decides to advance a drug to the next stage of development, it's done because they think there's a sufficiently profitable market at the end of the road. So lots of extremely promising drugs stall out if the numbers don't make sense. Furthermore, the trial enrollment process itself is cumbersome and often your only entrypoint is email "[email protected]" and hope you can a response and can navigate their arbitrary screening process. Speaking of screening: trials are full of exclusion criteria. They can get copy-pasted between trial protocols and often have no real reason excluding you. What's the washout period for your last line of chemo? Ever had any kind of immunotherapy? &c &c From the point of view of whoever is trying to get a drug approved it's better to be safe than sorry wrt any kind of uncertainty in a patient's medical status or treatment history. But that means that there might be a drug that looks extremely promising from previous trials but the only current trial excludes you for a small reason.

"Sorry, you can't buy cars privately any more -- you need a gov't agency to make sure you don't get screwed. You cannot fend for yourself, right?"

Similarly, you already cannot buy all cars privately anymore. Business-to-consumer is highly regulated (that's what the Federal Motor Vehicle Safety Standards are for, but home-built cars also need to pass several inspections before they are allowed on the road.

Taking a risk just driving a car, I don't get why you can't take a risk with a medicine.

This also harms eveyone because all the money pouring into a treatment that was a scam from day one is oppertunitu y lost.

You guys remember Theranos? When Safeway announced their deal with Theranos in 2012, a bunch of companies working in blood diagnostics folded because they couldn’t raise money or because their investors clawed back entire rounds.

A couple of friends had just raised a series B round to commercialize their surface plasmon resonance blood testing device, which would have actually done what Theranos promised with onsite blood testing (except with proper blood draw, not a finger prick). They were getting the product ready for golden standard testing with a CRO when investors decided it wasn’t worth trying to compete with a Theranos already in the pocket of major pharmacies and pulled the series B funding.

Patents ended up getting sold to some patent troll shortly before the news broke that Theranos was a scam.

The people who funded Theranos weren’t even desperate, just greedy. Imagine what fingers they’ll put on the scales when they’re actually staring death right in the face? It’ll be a disaster.

--

[0] - The situations can be ethically and emotionally complex, so I'm not passing any generalized judgement, other that this is something a person might be concerned about when contemplating the possibility of ending up in this situation in the future.

[1] - Arguably the same thing, but that's another topic.

Bad money chases out good, increasing the fraction of money in bad hands by billions or tens of billions is going to have a palpable effect on the stability of the world.

Selling straight up snake oil is still fraud, and letting money be thrown at overhyped but ineffective treatments is a smaller tragedy than forcing people to die when treatments exist.

I trust the Chinese government less than I trust my own with regard to doing the right thing (just because it is the right thing or indeed at all), and that is a pretty low bar…

https://en.m.wikipedia.org/wiki/Death_of_Wei_Zexi

Then there was some movement on the issue in 2019:

https://www.bloomberg.com/news/features/2019-12-10/cancer-tr...

I’m not sure where they ended up due to COVID happening next.

There's a difference between something that works at least in a petri dish or mice, and something with no plausible mechanism of action like homoeopathy.

It would be a much better case for regulating if there were some external payer.

You can't have freedom without some chance of people making bad choices.

Dad asks me for a lot of money for a snake oil cure that might prevent him from dying from bile duct cancer in a month or two…how am I going to say no to that?

I have a severely disabled dad and have had this convo.

https://timharford.com/2020/06/cautionary-tales-the-spreadsh...

it seems like you are contradicting yourself. if it's likely to cure, then that would be borne out by the trial? And then it would be rushed for approval. The vast, vast majority of clinal trials for cancer are unsuccessful. A 'successful' drug may add a few months of life vs placebo at a cost of $10k or more per month.

I'm not sure you know how long these things take, even when they work well.

Also, the failure to show a positive clinical effect in a trial mean one doesn't exist, especially at the individual level.

I was finally able to get it online. I no longer take pain medication. Life is basically back to normal.

https://sciencebasedmedicine.org/the-cruel-sham-of-right-to-...

> So-called “right-to-try” is a cruel sham that holds out the false hope of survival to terminally ill patients and their families. In return, all they have to give up is patient protections and agree to pay to be guinea pigs to test a drug company’s product. The product of an ideology that uses the terminally ill as shields to hide the ideological motives behind the law, which are to hobble the FDA, right-to-try is a terrible idea.

No one would opt into volunteer to be in the second group. But, I can see a lot of, e.g. 50-year-olds, taking a risk thinking they can refill the college-education funds and retirement funds by just working longer than they intended.

I would imagine when dying one of the comforting thoughts is that your estate will continue being able to care for loved ones after you are gone.

But in general for the larger point, we should make sure that people can fight to live, but that assholes aren't stealing from them by selling magic beans.

With the modern social safety net, perhaps the traditional protections can be relaxed.

Sure, why couldn't it be? Maybe it has severe, painful side effects. There's a reason many people with stage 4 cancer and low chance of survival decide not to go back on chemo. Maybe it kills them earlier than expected. Then what?

Of course some here would view that as a positive.

If there really is a treatment that is significantly better than current treatments, have your doctors talked to you about this? It was codified into law about a decade ago, but that's not so new that they shouldn't be aware of the program.

You could find out which centers close to you are part of the trial, and contact them about getting access to treatment.

I've previously contacted a drug manufacturers directly to inquire about access via a trial center. This was for a pancreatic cancer drug made by Halozyme. The trial was ultimately halted due to disappointing results.

Please take into account that a drug still in trial may ultimately fail to meet its endpoints, as in the above case.

[0]https://www.fda.gov/news-events/public-health-focus/expanded...

BioNTech has a similar treatment that's also been apparently slow to launch Phase III trials.

First, human beings aren't Guinea pigs. Yes, the FDA is slow - for damn good reasons. We didn't use to do things this way and then we changed it to fix problems. There are reasons things are the way they are and it's very well-understood in the medical field. Medicine is not a move fast and cause pain and suffering kind of field or move fast and kill people kind of field. They're very conservative.

Two, since human beings aren't Guinea pigs when we do do experiments on them with drugs that have been through a very thorough and extensive pipeline already, they're looking for very specific things. This tends to make a very small population of patients eligible for a trial.

Three, trials are expensive. These aren't drugs that have been scaled-out in production. The costs tend to be exceedingly exorbitant. The medical field is very well aware of the inelastic demand for their services and this opens up new avenues for the unscrupulous to prey on people. Like the used to do in the "good ol' days."

Finally, the fact the treatment you would like to pursue is still "stuck" in clinical trials is indicative they're not convinced of the efficacy of the treatment. I can tell you lots of "promising" treatments never make it out of clinical trials. That's the nature of the beast. Most ideas are dead-ends.

My advice? Don't chase unicorns. Sadly, I've watched too many people in my family die while chasing unicorns and getting obsessed with treatments that are in trial. There are always treatments in trial. Live your life and enjoy what time you have to the fullest.

where he shows, with statistics, that moving fast saves more lives than it costs.

The problem with this is that they don't appear to be optimizing for the minimization of harm, which seems very obviously to be what they should be optimizing for. Instead, they're optimizing for the minimization of harm caused by new drugs/treatments/etc., while ignoring the harm caused by disease. It's all well and good to not cause suffering through your actions, but suffering caused by inaction (or slow action) is still very much suffering that's caused by your decisions.

Of course they are. From an ethics standpoint they're responsible for the harm caused to you by new drugs/treatments/etc. They're not ethically (or legally - but that's another matter) for nature harming you via the ravages of disease. If you're familiar with the education of anyone in a medical or medical-related environment then you know it's pounded in their heads to do no harm. Medical intervention is to ameliorate, not deteriorate. Medical intervention is not prescribed if it's not known to heal. That's when palliative care is called for.

I will say it again - human beings are not Guinea pigs, especially when they're dying.

I would argue that from an ethics standpoint they're responsible for the results of their decisions. If they hold a treatment up for an extended period of time because it might have some complications, but with the benefit of hindsight it is clear that it would have saved lots of people who instead died, they bear some responsibility for those deaths.

> If you're familiar with the education of anyone in a medical or medical-related environment then you know it's pounded in their heads to do no harm.

As it should be, but the point is that harm can be done by inaction, not just by action.

Due to the lack of control groups them taking the pill does not make the science go forward (in the sense that it is an indication for the use).

I believe however that they're is a great interest in the stopping effect: the evidence that the treatment is ultimately harmful.

Still, this is a win-win:

- if the patient gets better, good for them. Does not help the pharma company.

- if the patient gets collateral negative effects, too bad for them (but they were very bad anyway). This helps the pharma company.

Coupled with euthanasia, this would be a humanitarian solution for the extremely sick.

It may also be that on day 6 you get better (no matter of you took the pill or not), and then worse on day 20, but much worse than without the pill.

Still, this is a statistical wink so to speak (paraphrasing xkcd (https://xkcd.com/552/)).

To be clear: I am all for patients experimenting on themselves when they do not have much to lose (and whenever they please, but they have to take their responsibilities then). Plus euthanasia if the experiment goes south.

In some cases this is true, but people often want to risk their chance of survival with conventional treatments.

In our current society, a doctor has the unilateral decision whether to end a patients life (in so far as they are able to thrive in our society). Access to stimulant and opioid medication should be a right for patients with these conditions.

We don't need a doctor to baby us, when they would certainly have access for themselves and their children. The majority of doctors would not give up their career, move in with their parents, or tell their kids that college is not for them.