> given that all of the nginx files are not in a single directory it's my understanding that you'd need multiple COPY commands

Workaround would be to move these files to one directory on host, COPY it in one command to /tmp (or even better, /dev/shm or other ramdisk) and then use script to distribute files where needed.