News for hackers. Stuff that matters.
Re:Better and Faster Large Language Models via Multi-Token P... by namibj on Wednesday May 01, 2024 @09:54AM (#40223287)
The problem is that a position's probability output is conditioned via attention on all previous positions.

If you want to be better you need to switch to DDPMs for example (e.g. an encoder-only transformer to predict diffusion transition probabilities in parallel, then apply steps of denoising).

The problem is just that these don't work so well from auto regressive decoder transformers, and encoder-decoder architectures like e.g. Google's T5 have fallen out of favor since about LLAMA dropped.

There are no comments on this comment yet.