Re:Diffusion Models Are Real-Time Game Engines
by crooked-v on Wednesday August 28, 2024 @06:16PM
(#41385030)
Give it state by having a rendered-but-offscreen pixel area that's fed back in as byte data for the next frame.
Re:Diffusion Models Are Real-Time Game Engines
by TeMPOraL on Thursday August 29, 2024 @05:17AM
(#41388874)
Huh.
Fun variant: give it hidden state by doing the offscreen scratch pixel buffer thing, but not grading its content in training. Train the model as before, grading on the "onscreen" output, and let it keep the side channel to do what it wants with. It'd be interesting to see what way it would use it, what data it would store, and how it would be encoded.
Wouldn't be as pure though.