News for hackers. Stuff that matters.
Re:What We Know About LLMs (A Primer) by jawerty on Tuesday July 25, 2023 @06:39PM (#36870686)
I see, that’s my misunderstanding I was grouping all training as pretraining
Re:What We Know About LLMs (A Primer) by wilhelm____ on Wednesday July 26, 2023 @08:39AM (#36876667)
pre-training is developing the language model's base understanding of conditional word probabilities.

SFT and RLHF is attempting to further guide the model in terms of steerability + alignment of output.

In fact, the InstructGPT authors were worried about losing the pre-trained model's underlying probability distribution, so they try a version where it penalizes the model deviating too significantly from the original distribution (using KL). I don't remember them seeing a significant difference in performance.