Re:DiffusionGemma Technical Report
by LarsDu88 on Thursday August 20, 2026 @02:05PM
(#49378067)
Denoising is probably the weakest part of this model. There is recent research from Kaiming He showing that predicting the noise is actually not the best strategy for image generation since noise space is so large.
Simply predicting the surface of the data you are trying to generate is far more representationally efficient, and perhaps in the next few months we'll see a version of that for LLMs
Re:DiffusionGemma Technical Report
by yorwba on Thursday August 20, 2026 @02:21PM
(#49378260)
Diffusion language models work with a discrete output space, unlike image models that repeatedly refine a continuous output, so they don't do the noise-prediction thing anyway.
Simply predicting the surface of the data you are trying to generate is far more representationally efficient, and perhaps in the next few months we'll see a version of that for LLMs