This post https://twitter.com/madebyollin/status/1566838643771457536 is a step in such direction. I quite like the author's vision about neural nets eating through the graphics pipeline, given there are work like Gancraft (https://nvlabs.github.io/GANcraft/) and GTA photorealistic GAN (https://isl-org.github.io/PhotorealismEnhancement/) have produced stunning results.