News for hackers. Stuff that matters.
Re:Language Models Are Super Mario: Absorbing Abilities from... by skybrian on Saturday April 06, 2024 @01:02PM (#39953726)
The weights need to be connected to something interesting. Pre-training is how they get them all connected up, and fine-tuning is how they find which weights it would be useful to change.
Re:Language Models Are Super Mario: Absorbing Abilities from... by stoniejohnson on Saturday April 06, 2024 @02:29PM (#39954368)
Pretraining determines the weights (e.g. the connections), fine-tuning let's you change some subset of the weights (e.g. the final layers) with a smaller chunk of task-specific data.