With a service it's all much easier because you can have pretty robust integration tests and you're good to go. With an ML model it's a lot trickier because even if you have sample payloads and responses there might be other payloads you didn't think about that could cause issues. Or it might rear its ugly head when you finally retrain the model and suddenly model performance is crap.
One recent example that comes to mind is that PyTorch now disables Ampere GPUs' TF32 units by default since there were some hard to find edge cases where the matrix multiply results were completely wrong compared to the expected result. This went mostly unnoticed except in a few cases where users were trying something a bit more sensitive to such issues.
I had a similar issue in one of my own models where there was an error in a normalization layer's implementation in Tensorflow (it was computing the average over the wrong axis iirc). It only became noticeable to me when I came back to retrain the model a few months later with an updated Tensorflow version and found that the results from training were not even comparable.
In every other ecosystem when minor version are updated there are very rarely issues, and there are little issues with compatibility with minor versions of libraries.
In part due to a culture that encouraged monkeypatching. With the result that what behavior you get depends on who last monkeypatched it. So a dependency 3 deep that loads the module you expected to load last suddenly causes the monkeypatch you were depending on to disappear...
If you are willing to run other people's arbitrary (Python) code, then - vulnerability or no, you should consider the system running this code as compromised.
The issue: ML workloads are very sensitive to the exact version of a framework/library used and these frameworks/libraries are not stable despite what their semantic version says.
So what do you do when you have a python vulnerability? Best case, an update "just works" but in many cases you either need to retrain the model (can be extremely expensive or inconvenient), rewrite it if APIs changed, or hunt down a difference in model output if some downstream dependency broke something. Stability of software in the python world is abysmal.