Delete the entire docker stack. It's utter shite. Then install Podman and Buildah, and rejoice! For all the evil buildx magic can now be done explicitly.
This solution was born when trying to do native builds on CircleCI - where you do not have a buildx cluster with multiple architectures. You have to build the archs as independent build steps, then pull them together afterwards. This is something that Docker never considered, and Docker only functions on the happy path. Buildah has multiarch manipulation commands: https://danmanners.com/posts/2022-01-buildah-multi-arch/ (not a plug, first blog on search)
We migrated to GHA, but still use separate builders to avoid the Docker trite. We also ditched DCT (Docker signing) for Cosign because - again - DCT is home-grown garbage.
We ended up switching to using multiarch/qemu-user-static:7.2.0-1 <https://hub.docker.com/layers/multiarch/qemu-user-static/7.2...> which also mutates binfmt_misc in buildx's context in order to exec the static copy of qemu in it but I find their shell script much more legible about what's going on: <https://github.com/multiarch/qemu-user-static/blob/v7.2.0-1/...> not to mention the fact that it more clearly represents the recent qemu version versus secreting it away inside the moby/buildkit:heheh-good-luck-friend
They have an entire GH label for that: https://github.com/docker/buildx/labels/area%2Fqemu and my journey into the buildx sewers started from https://github.com/docker/buildx/issues/1170