This is the first I've heard about serious performance overhead from going rootless. Do you have any links with more info about it?

I haven't encountered any issues like this personally with rootless podman (although I'm not doing any large scale deployments).