Any dummies explanation of what diffusion is?

I've since moved on to work primarily on diffusion models, so I have a series of blog posts about that topic as well!

- https://sander.ai/2022/01/31/diffusion.html is about the link between diffusion models and denoising autoencoders, IMO the easiest to understand out of all interpretations; - https://sander.ai/2023/07/20/perspectives.html covers a slew of different perspectives on diffusion models (including the "autoencoder" one).

In a nutshell, diffusion models break up the difficult task of generating natural signals (such as images or sound) into many smaller partial denoising tasks. This is done by defining a corruption process that gradually adds noise to an input until all of the signal is drowned out (this is the "diffusion"), and then learning how to invert that process step-by-step.

This is not dissimilar to how modern language models work: they break up the task of generating text into a series of easier next-word-prediction tasks. In both cases, the model only solves a small part of the problem at a time, and you apply it repeatedly to generate a signal.

Yes: https://github.com/riffusion/riffusion

yeah, we're training a diffusion model for this here: https://sonauto.ai/

example 70s dolly parton song made by a rando earlier today: https://sonauto.ai/songs/t4XGcHqsYwS6B73MjtV5

I hope this isn't a sign of what is to come in music.

We can do anything imaginable with AI and instead we just make really shitty Dolly Parton music reboots.

Maybe we can do AI Tom Jones too and form a whole genre of shitty AI 70s music.

Agreed. The above is essentially a tech demo while we work on the more exciting/fulfilling uses we're planning for these models.