What's the CPU arch of the container images you're running?
The postgres image (postgis/postgis:12-3.1-alpine) is linux/amd64.
I see now that the official postgres images have other arch options compared to the postgis images, which are arm64 only. I wonder if that could make a difference...
And now I see that Docker Desktop actually shows a warning about potentially poor performance when using an amd64 image: "Image may have poor performance, or fail, if run via emulation."
This is good news, since I'd much rather use Docker to run postgres in dev.
Correction: in paragraph 2, "arm64" should be "amd64"
Update: I switched to an official postgres image and it works great. I just had to create a derived image that installs PostGIS like so:
FROM postgres:13-bullseye
RUN apt-get -y install postgresql-13-postgis-3
On my M1 Mini, I found it unstable and incredibly slow (for Postgres at least, using stock images from Docker Hub) and mostly quit using it. I upgraded to the latest version recently and it was still too slow.
For everything else, though, I find that the M1 Mini is significantly faster. E.g., compiling Python versions is way faster.