For example: a SPA webpack setup using html-webpack-plugin which generates the <script> tags in index.html for the different assets, later served by nginx or similar.

How does one tell nginx the names of the files to push if they are generated dynamically? You have to put the names in a separate file and then somehow configure the server using that file?