If there’s (hypothetical overly high numbers here) a 10% chance of dying from a disease or a 1% chance of dying from the vaccine from that disease, I want the vaccine and would like the freedom to do so. But if a new vaccine is coming out and soon after we see a 1% death rate from it, now I’m gonna want to hold off and take my chances with the disease because that 1% may turn out to be as high as or greater than 10% down the track when more is known.
The main complexity here is just like the early days of coronavirus - until things have played out for a while you can’t have confidence about the extent of the harm and that’s true for both the disease (which we now have over a year of information about) and the vaccines (which we have a month or two of in-the-wild info so far).
The point here is simple. It's worse for someone to die of a vaccine than for someone to die of the coronavirus. How much worse is up for debate, but it's a significant factor and napkin math has to take that into account or it's morally bankrupt.