In support of your argument; Look for example at the Dockerfile for the official Golang container. They pin exact sha256 hashes for each architecture, and the source release in case you're on an un-binary-released architecture.

Pin specific versions of your packages, coupled with caching and you're sitting pretty.

Yes, but you still can't guarantee anything about the host the docker container has to run on, so you're still impacted by host configuration and therefore still need to provide a good base environment for running your containers. This is fairly simple with most providers, but in such a case, using Ansible or similar to deploy directly to the host has similar results.