Over the years I've become a little obsessed with imagining the ways things can fail (I'm just as fun at parties as you can probably imagine).

An example that immediately jumps to mind where that is not true for software would be something that quietly uses some resource that you didn't expect (open ports, filehandles, space on disk/memory, the space inside of however many bits you have for a field, anything like that). These are things that you don't know to test until you've seen them fail, and only then can you start to automate.

One possible analogy here is cancer or autoimmune issues. If you get diagnosed with either of those today, they almost certainly started a while ago, they just weren't detectable until today. Or maybe they would have been if you knew exactly where to look, just like if you knew to keep an eye on how many ports you had left on your live server. (Not claiming these vaccines cause these things, I'm sure some peoples blood pressure started rising as they read this).

The complexity of software is nowhere near that of the human body or populations of humans, and yet it can be incredibly hard to predict. Multiple backwards compatible versions on the client and server side, running on different hardware with different operating systems, all of these things changing over time. Unexpected behaviors can absolutely show up at any time.

Of course you can't test it all, but you do what you can, and then after that you monitor very carefully, but its way more expensive to fix the problem after it reaches prod. In humans, there's a chance that its not just expensive, it's impossible.

Some engineer creates a fix by turning off some recently added experiment responsible for the service degradation. The rollback passes all unit tests and integration tests. You know this fix is low-risk (maybe because your service was healthy for the ~1 year prior to the rollout). Even if there is some risk that this rollback breaks the service at some point in the future, in comparison to the known revenue loss the expected loss of that breakage would be minimal.

Another engineer objects to the rollback saying that "they don't know of the long term effects of such a rollback." You try to counter their claims by showing that previous rollbacks were almost always successful, and when they caused another failure that failure was not nearly as bad as the original service degradation. Also, the rollback passes all tests. That engineer then repeats "we don't know of the long term effects of this particular rollback, and I'm skeptical that the fix is good because the failing experiment and rollback was identified so quickly."

(Rollback vs. fix-forward doesn't actually matter; the point is that all available knowledge shows that the fix is low risk, historically supported, and that there is likely no mechanism for random failure in 6+ months.)

Do you think it would be correct to prefer the second engineer's claims?

But the assumptions that you use to calculate that net are where all of the disagreements stem from. In your example, you've constrained it to losing well defined revenue, and "you know this fix is low risk", so the answer is easy.

Maybe (wrongly or rightly) you don't have faith in the revenue numbers, or in the test suite or the engineers writing the tests or the fix, maybe there's office politics involved, etc.

This thread started with my reaction to these lines from op:

> Knowing long-term effects (1+ years) is not a precondition of approval for most vaccines. It just so happens that vaccines usually take a long time for approval, so we end up knowing long-term effects as a side effect. This is because there generally isn't a known mechanism for vaccines to have long term effects. Not ruling out the possibility, but we understand how vaccines work pretty well. [...]

I've seen this claim dozens of times, and it always always always makes me think of those scenarios where you find a bug in code that you or someone you trust/respect wrote from the ground up, and you wonder "how in the f*** did this ever work?". I'm not claiming I'm right to feel that way, maybe this is a defect in my character, but I just can't buy the "we understand vaccines pretty well" thing.

You seem to be taking this from the perspective that, because there are unknowns, literally anything could happen. And we don't think that's the case with my vaccines or code.

Pushing v2 of your webapp to master isn't going to cause rockets to launch. There are things that could happen and there are even very surprising things that are unlikely but could happen. But not everything is possible.

When we say "we understand how vaccines work pretty well", we mean that in the same was as "we know how our app works pretty well". That doesn't mean that we can't still be surprised, but it does mean that we have some things that we can be confident won't happen.

One such thing is sudden onset symptoms well after the thing has left your system, because there isn't a biological way for that to have an effect (to use the code analogy, you enable an experiment, it modifies database writes for a while, then you disable the experiment and delete the code. Once you've deleted the code, we know it can't do anything else, because it's not there anymore.

That's true with both vaccines and webapps.

The hard part about conversations like this are that you could have written the definitive book on this stuff, or you could be just another dummy on the internet like me. So I have to take a statement like that and do my best to evaluate it myself, since I have no idea how to value your expertise. If you're an expert of some sort, I apologize in advance.

We know for a fact that vaccines have long term effects. They wouldn't be useful otherwise. It would be nice if they could only have positive long term effects, but why would there be any guarantee of that?

Symptoms could start immediately but be undetectable until they progress far enough (think a tiny little tumor for just one example). Again, not claiming this is happening, just saying that it seems nutty to assume it can't happen.

When I read "we understand how vaccines work pretty well", I automatically translate it to "we understand pretty well what happens when we inject a substance into a human body", and then I think "really?". We have some useful models, sure, but "understanding pretty well" seems a vast overstatement of the situation, especially when its laypeople chatting on the internet.

The abstract concept of a vaccine is pretty simple. Anybody can regurgitate the picture book version of it. Any specific instantiation of one is remarkably complex, all the way from manufacturing, to transport, to administration, and then interacting with the legacy ball of spaghetti that is the human body and it's various systems.

The fact that we humans decided to name some substance the same thing that we named some other substances that we created in the past doesn't necessarily tell you how that substance is going to interact with the body. It can provide you with hypotheses, but hypotheses need to be tested.

Correct, but, and this is the important bit: the only lasting thing the vaccines do is leave your body primed to react to certain proteins (everything else they "do" leaves the body pretty quickly, you can think of this as a fairly limited api they're interacting with). This could be a bad thing, but if it is

1. Anyone who naturally caught covid would also suffer the same consequences

2. you'd expect to see the issues as soon as the immune system was primed, this takes days to weeks, not months.

So yes, there are all sorts of weird thins that could happen, but I'll reiterate: we know of know biological mechanisms that would make those things happen suddenly, without warning, months after the vaccine. We understand, very well, that such things can't really happen.

Just because you don't understand the human body very well doesn't mean the medical establishment also has equally no clue.

That is of course the goal, and assumes nothing goes wrong in the manufacturing, transport, or delivery mechanism for the payload. But if they haven't been tested for long term adverse effects, how do we know this? And again the answer ends up being because we know how vaccines work and round and round we go.

But let's leave that and assume that it actually does verifiably only prime the immune system to react to certain proteins. Is that some small thing? Immune disorders can range from annoying to horrific, and they're all from just reacting to certain proteins.

> This could be a bad thing, but if it is 1. Anyone who naturally caught covid would also suffer the same consequences

This seems to be a hypothesis. The current vaccines are based on a modified little chunk of a now extinct(?) variant of the virus plus some novel packaging. That they should have the same effect on the body as the whole current or future dominant variant (edit: entering the body via a different pathway) doesn't seem obviously true.

> you'd expect to see the issues as soon as the immune system was primed, this takes days to weeks, not months

Why? I'll use the same example again. If a tiny little tumor were formed, or would be formed over time as a result of some protein reaction that started right after being dosed, why would you expect it to be detectable immediately?

The specific mechanism isn't important, it's just the "we can't think of a way it could fail, therefore we dont need to verify this" is just impossible for me to get my head around.

Feel free to drop the conversation if this gets frustrating. I will continue to ask until I understand, and I may not get there. I'm actually enjoying this, but I know I'm weird and this drives some people nuts.

> Just because you don't understand the human body very well doesn't mean the medical establishment also has equally no clue.

Fair, but that's not my claim. I can be at 1, while the medical establishment is at 100, and actual understanding of what's going on in the body could be 10000. We do some black box testing and get some somewhat repeatable results. That's a pretty amazing feat, but still a far cry from really understanding what's going on under the hood.

Pushing webapp code to master certainly won't result in rockets launch, if only because no sane person physically connects launching platform to build server ("no biological pathway"). On the other hand, deploying build artifacts carries much more risk. This risk is usually mitigated by extensive multi-level testing and appropriate live monitoring. In addition, the target environment for software is, at least theoretically, fully tractable. I.e. it can be de-constructed and understood to the very last bit. It is also fully human-made and controllable, without random nuclear reactors hooked up to web server. If such invariants cannot be established, I think it is not ethical for a senior software engineer to approve mandatory wide-scale deployment. Under such conditions the decision must be strictly on the owner of each server.

In case of vaccine we deal with infinite number of reasonably unique "environments" - human bodies. We know that each of these "environments" are absolutely capable of developing a million of adverse things like cancers, strokes, autoimmune diseases, etc. So, the "biological pathway" exist, the "rocket launchers" are there. Our understanding of inner workings of human body is very imperfect, as demonstrated by the fact that humans still suffer and die from all sorts of diseases. Our monitoring tools are very limited, and our analytical framework is just blunt statistics. The vaccine is, basically, a clever hack into a half-understood system. So, it is certainly up to the owner of the body to decide what to "deploy" into it. And disagreement or misunderstanding, in my opinion, is exactly around the concept "who really owns one's body".

The database analogy is also interesting, in that while the experimental writing code is gone, the database remains modified. In sufficiently complex and poorly understood system how can one guarantee that modified data doesn't cause any unhandled exceptions down the track, especially given the number of vulnerable third-party clients?