Re:Language Models Are Super Mario: Absorbing Abilities from...
by skybrian on Saturday April 06, 2024 @01:02PM
(#39953726)
The weights need to be connected to something interesting. Pre-training is how they get them all connected up, and fine-tuning is how they find which weights it would be useful to change.
[ Parent
| on HN ]
Re:Language Models Are Super Mario: Absorbing Abilities from...
by stoniejohnson on Saturday April 06, 2024 @02:29PM
(#39954368)
Pretraining determines the weights (e.g. the connections), fine-tuning let's you change some subset of the weights (e.g. the final layers) with a smaller chunk of task-specific data.