Unless and until state can be stored outside of the model.

I for one would love a video game where you're playing in a psychedelic, dream-like fugue.

It's kinda crazy though that a single game session would be burning enough natural gas to power 3 cities. Unless that's not true

It is plausible to run a full simulation the old fashioned way and realtime render it with a diffusion model.

It is not currently, or near term, realistic to make a video game where a meaningful portion of the simulation is part of the model.

There will probably be a few interactive model-first experiences. But they’ll be popular as short novelties not meaningful or long experiences.

A simple question to consider is how would you adjust a set of simple tunables in a model-first simulator? For example giving the player more health, making enemies deal 2x damage, increasing move speed, etc etc. You can not.

Unless you can change fundamental problem that it doesnt follow the state the way you want exactly