How I solve this is that I deactivate all configuration files that have missing certificates, and then reactivate them again after the files have been created.

https://github.com/JonasAlfredsson/docker-nginx-certbot