Had similar issues with podman on a Steam Deck of mine that I use as a little home server - eventually got a configuration working fine but was a real pain.