News for hackers. Stuff that matters.

A comment on Transformers on Chips. Parent comment

Re:Transformers on Chips by Me1000 on Saturday December 16, 2023 @08:03PM (#38669410)
This was my first thought too. Even if transformers turn out to be the holy grail for LLMs, people are still interested in diffusion models for image generation.

I think we’re about to see a lot of interesting specialized silicon for neural nets in the coming years, but locking yourself into a specific kind of model seems a little too specialized right now.

[ Parent | on HN ]
Re:Transformers on Chips by dontwearitout on Saturday December 16, 2023 @09:03PM (#38669719)
Diffusion models could actually be implemented with transformers, hypothetically. Their training and inference is what makes diffusion models unique, not the model architecture.
[ Parent | on HN ]