I suggest you start by just getting plain podman running the docker.io/hello-world container to reduce the complexity and simplify debugging if anything goes wrong. It's been about a decade since I last touched windows but if wsl2 is 1:1 with Linux the official podman guide should be straight forward.

It's always easier to start with the bare minimum and build from there, and you will get a better understanding of the tools you're using.

I don't know what `vscode devcontainers` is but to run podman on wsl2 I simply installed a fedora wsl2 image (by importing the fedora container image if my memory is correct).