Yes. Then add a service on the OS that fetches the libraries for any binary. It's not a complicated thing. It's been obvious to me for years that this is how exact version linking should work.

Linux at least has library versioning so this is less of a problem. Windows has foo.dll and foo.dll and foo.dll, so pretty much the only way is to ship your own foo.dll.