Pretty much. I've been able to use some regex-replacements on the build output in the past with varying luck. But it definitely feels like a hack and can break in weird ways. Another option is building all environments into a single container image and then selecting the right build runtime using an environment variable. This makes the build costlier, even if you can build in parallell, and a bigger image takes longer to distribute into your environment.

At that point dedicated frontend hosting infrastructures like Netlify, Vercel, Cloudflare Pages and others seems like a better choice.