Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Large language models are having their Stable Diffusion moment. Parent comment
> These nets regurgitate the most average possible output given inputs. By definition.
You seem to be confusing the training objective with the capabilities.
For example:
> Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
https://arxiv.org/abs/2212.10559
But note that this is happening at inference time, so it's not just generating the average of the input the model is trained on.
> These nets regurgitate the most average possible output given inputs. By definition.
You seem to be confusing the training objective with the capabilities.
For example:
> Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
https://arxiv.org/abs/2212.10559