Big fan of pipx, but I don't think it would work here? Local ansible calls remote python directly and that's where the modules need to be, so a sandboxed {local,remote} ansible would be inert.

There's a workaround but it's not ergonomic and I always forget what it is.