You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients. The numbers are so large that other factors (environmental, genetics, etc) cancel out. So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation.
(Full disclosure: my family used to own a company that ran clinical trials for pharmaceutical companies)
Yes, you can do this. No, it's not the same.
With this kind of uncontrolled, longitudinal study, yes, some things are quasi-controlled: genetics, possibly lifestyle, other drugs, etc. Anything that plausibly doesn't change (much) in a single person from time A to time B.
Some things you can't control for, but still matter: changes in behavior due to the event itself. Placebo effect. Sample bias (e.g. the patients receiving treatment X were selected to receive treatment X because it was felt that they would benefit from it. This is subtle, but can really mislead. In the example of vaccination, imagine that the people most likely to vaccinate their young kids are also the ones most likely to keep their kids isolated at home, in a protective bubble...)
> So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation.
The key part of that is the last three words: worth further investigation. To get the final answer, you still need a controlled experiment.
So the moment we knew the vaccines worked, the science got harder. Well, that point also save an enormous amount of lives, so I don't think this is something to complain about. It's still possible to do a lot of good science now, even randomized controlled trials. They just can't compare to unvaccinated groups, but you can still do stuff like compare double-vaccinated with triple vaccinated.
You could easily find millions of potential study participants who don’t want the vaccine. The study wouldn’t be double-blind, but it would be single-blind and better than what they chose to do.
There is one study that listed negative effectiveness against Omicron for the vaccines. That study did not control for any confounding factors, it simply was not designed to measure this particular thing, it was focused on a different question.
There are some strong fundamental reasons why we would not expect the vaccine to be harmful with future variants. And even then we would still be able to detect if it was.
That's the point! That's why we need controlled trials. With an unvaccinated group. You know, the thing that you say we can't have, because it's unethical.
> ...without adjustments.
As usual, there's an XKCD for that:
> You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical.
I see this position put out there a lot and generally go unchallenged. For the record some people think the opposite: that it was unethical to unblind the placebo group so early. At the time you could estimate that continuing the placebo group might lead to ~60 unnecessary deaths from Covid-19 and a few hundred more serious cases. That is in the lucky scenario where the early success held. But in all scenarios, in return for a few dozen volunteers risking their lives, billions of people would benefit from randomized, placebo-controlled trial data which the NIH itself calls the "gold standard" as to "whether or not a treatment is safe and effective".
IMO the harms we have gone through from flying in the dark (people underhyping/overhyping the vaccines), multiplied by the number of people involved, make this a case where unblinding was unethical. It is very easy to imagine that far more life was lost in the general population from being in the dark than was saved by unblinding the volunteer study population.
(There's also the issue of whether or not it would have been practical to keep the participants blinded. I think that's a challenging topic of its own orthogonal to the ethical question.)
We may have loads of data from real people being treated, but good luck trying to analyse it in a scientific way.
We need effective vaccines, and we need to be able to trust them. Just treating people with a vaccine doesn't always give that trust and confidence.
(I hope I got my medical terminology right, I'm not a clinician, but have been close to covid-19 related medical research)