News for hackers. Stuff that matters.
Re:Diffusion Without Tears by getnormality on Friday February 14, 2025 @05:28PM (#43053807)
I've noticed that when finance guys learn that there's a really useful AI thing called diffusion, they get all excited and start writing stochastic differential equations and drawing 1-D Brownian motion plots all over the place. It's not yet clear to me whether this helps anyone understand AI diffusion models, but they seem to enjoy it a lot.
Re:Diffusion Without Tears by markisus on Friday February 14, 2025 @05:39PM (#43053861)
If by helping anyone you mean helping the math challenged (which is technically almost everyone) then I would be inclined to agree.

But to the quant crowd I’m guessing that couching diffusion in the language of stochastic calculus is helpful.

Re:Diffusion Without Tears by patrick451 on Friday February 14, 2025 @09:01PM (#43055136)
I'm a controls guy, not a quant guy but I've found the SDE perspective and this blog post to be incredible in helping me understand how a diffusion model works.
Re:Diffusion Without Tears by currymj on Friday February 14, 2025 @09:32PM (#43055323)
it makes intuitive sense, but at the same time, you can get good performance replacing the Gaussian noise with deterministic transformations.

so in the end diffusion can’t explain that much about how diffusion models work. disappointingly, so many modern ML things are like this.

Re:Diffusion Without Tears by sl8r on Saturday February 15, 2025 @03:37AM (#43056955)
Hey! I’m the post author. Highly recommend Sander Dieleman’s blog for alternative interpretations https://sander.ai/2023/07/20/perspectives.html

I personally find the SDEs the most intuitive, and the deterministic ODE / consistency models / rectified flow stuff as ideas that are easier to understand after the SDEs. But not everyone agrees!

Re:Diffusion Without Tears by currymj on Saturday February 15, 2025 @10:53AM (#43059385)
Thanks for sharing this! I tend to agree that it’s easiest to understand this way.

I just find it a frustrating fact about modern machine learning that in fact, the nice SDE interpretation is somehow not the “truth”, it’s just a tool for understanding.