You can scream at the developers that they've violated semver but a "bugfix" is entirely subjective (relevant xkcd, spacebar heating, etc).
And even when developers violate semver in a point release the problem still exists. They actually rarely, if ever, rollback with a 1.0.2 that is equivalent to 1.0.0 and instead usually move forwards.
And if you have a language that supports loading 1.0 and 1.1 then there's no point in being artificially constrained over which two versions can be loaded at the same time based on the label, the underlying framework shouldn't be built to care. There's no need for a multi-version library loader to care about what a bugfix is.
Semver would just be an artificial impediment at this level.
It doesn't contradict SemVer.
However when you do that, it becomes a big deal to jump from one version to another even if the breaks are minor. So pros and cons.
So say there is v1 and v2 of a utility lib in my dep tree, but actually only using func A from v1 and func B from v2. Then I just have the AST of v1.A and v2.B in my deps and everything works.