Diffusion Language Models Are Super Data Learners

Re:Diffusion Language Models Are Super Data Learners by bicsi on Sunday August 10, 2025 @11:52PM (#44860626)

What if I told you that one can model bidirectional attention just by recurring over causal attention, and it’s still fast enough? Hint: It’s called chain of thought.

I strongly believe it’s time to discontinue diffusion models, solely on the fact that iterated auto-regression is faster, more parallelizable, and just as potent with proper prompting techniques (of course, unless you consider CoT as a form of diffusion, which it essentially is).

[ Parent | on HN ]

Re:Diffusion Language Models Are Super Data Learners by smus on Monday August 11, 2025 @12:34AM (#44860773)

Can you explain how CoT is a form of diffusion or models bidirectional attn?

[ Parent | on HN ]

Re:Diffusion Language Models Are Super Data Learners by fancyfredbot on Monday August 11, 2025 @04:53AM (#44862077)

I'd respectfully suggest that it's perhaps not time to "discontinue diffusion models". Minsky and Papert set AI back by decades by suggesting neural networks were a dead end which couldn't learn XOR. There's not a chance of an HN comment having the same effect of course but my point is that it's easy to dismiss things prematurely.

[ Parent | on HN ]

Re:Diffusion Language Models Are Super Data Learners by SalmoShalazar on Monday August 11, 2025 @08:01AM (#44863237)

Chain of thought is not a form of diffusion. Diffusion models clearly have characteristics that are useful and worthy of further research and should not be “discontinued“

[ Parent | on HN ]