No, that doesn't work. People make mistakes, and you end up not being able to build software. It might work _most_ of the time, but when things break, it's of course always at the worst possible time.
I think Cargo got it right: use semver, but also have lock files, so you can build with _exactly_ the same dependencies later.
This is nice, but should be noted does not absolutely prevent missing backward-incompatible changes; that a function has the same signature does not mean that it has backward compatible behavior.
(With a rich enough strong, static type system, used well to start with, you might hope that all meaningful behavior changes would manifest as type changes as well, but I don't think its clear that that would actually be the case even in a perfect world, and in any case, its unlikely to be the case in real use even if it would be in the ideal case.)
I've been bit by this a few times when I've come back to code 2-3 months later and forgot to include the .cargolock so I pretty much always use the lock file these days.
If that's your goal. There's a middle ground which includes security updates: asking the community to follow semver. Rust devs seem to do it well and my crates build well with non-exact version numbers after not visiting them for a while. Not sure why the majority of Rust devs do a good job with this and JS devs don't. I suppose it's the strictness of the contract in the first place.
With Go, I've found popular library maintainers doing a good job at BC contract compat even when there has been only one blessed version. I don't assume giving them a bit more flexibility is going to hurt anything.