ycy

It is hard to quantify the performance of samplers on individual images, but we have quantitative experiments on pretrained models in our paper (https://arxiv.org/pdf/2306.04848.pdf). Table 2 has a comparison to other samplers and Figure 9 plots the effects of the momentum term on convergence.

Glad you liked it! Just curious, what else would you like to see to make the post more complete? I had focused on getting through the basics of diffusion training and sampling as concisely as possible, but perhaps in future posts I can delve deeper into other aspects of diffusion.

The 2d toy models such as the swissroll takes 2 mins to train on the CPU, whereas the fashionMNIST model takes a couple hours on any modern GPU

Yes, these blog posts offer a different perspective on diffusion models from the "projection onto data" perspective described in this blog post. You can view them as different ways of interpreting the same training objective and sampling process. In our perspective, diffusion models are easier to train because instead of predicting the gradient of the _exact_ distance function, the training objective predicts the gradient of a _smoothed_ distance function. Sampling the diffusion model is akin to taking multiple approximate gradient steps.

To gain a deeper understanding of diffusion models, I encourage everyone to read all of these blog posts and learn about the different interpretations :)

My intention was to omit the details of batching in the blog post for clarity of exposition, and I'll update the post accordingly. Sorry for the confusion, but you can see the full implementation here: https://github.com/yuanchenyang/smalldiffusion/blob/main/src...

Author here, when I tried to understand diffusion models I realized that the code and math can be greatly simplified, which led to me writing this blog post and diffusion library.

Happy to answer any questions.

Nope, scheme is still being taught in the "Python class". The final project is writing a Scheme interpreter in Python, so students still get to learn Scheme.

http://www-inst.eecs.berkeley.edu/~cs61a/sp13/projects/schem...

I never stop discovering features in emacs, for example I found out about embedded calc mode yesterday. I was so impressed by it that I made a screencast demoing its features: http://www.youtube.com/watch?v=OPMgj3XWi8Q