Re:Scaling Transformer to 1M tokens and beyond with RMT
by cubefox on Monday April 24, 2023 @05:38AM
(#35685447)
For LLMs there are at least three different ways of "learning":
- Pre-training (for text prediction, using unsupervised learning)
- Fine-tuning (e.g. to follow instructions or to reject certain queries, using supervised and/or reinforcement learning) (optional)
- in-context learning (using "few-shot prompts" as examples)
Now the last two can have similar effects. For example, you might fine-tune a foundation (only pre-trained) model to follow instructions, or you don't and instead just modify your prompt such that it looks like a dialogue between a human and a helpful chat assistant.
But neither can replace the extensive pre-training phase, which is what gives the model all its intelligence.
One other disanalogy between fine-tuning and in-context learning appears to be that the model can't exactly remember the data it was fine-tuned with, while it "knows" exactly everything in its context window. That is its working memory, so to speak.
- Pre-training (for text prediction, using unsupervised learning)
- Fine-tuning (e.g. to follow instructions or to reject certain queries, using supervised and/or reinforcement learning) (optional)
- in-context learning (using "few-shot prompts" as examples)
Now the last two can have similar effects. For example, you might fine-tune a foundation (only pre-trained) model to follow instructions, or you don't and instead just modify your prompt such that it looks like a dialogue between a human and a helpful chat assistant.
But neither can replace the extensive pre-training phase, which is what gives the model all its intelligence.
One other disanalogy between fine-tuning and in-context learning appears to be that the model can't exactly remember the data it was fine-tuned with, while it "knows" exactly everything in its context window. That is its working memory, so to speak.