You don't need to rebuild the docker image and restart the container to get the extension installed & created.

And, I haven't found any Postgres docker image with all the extensions I need: you can reuse the RPM in dockerfile to build your own image, which always has to be done somewhere.