Re:Differential Transformer
by slashdave on Tuesday October 08, 2024 @01:28PM
(#41779686)
> You can just as well think of everything a model outputs as a hallucination
Exactly. Don't forget that an important factor in the success of GPT3 was RLHF, which is essentially training the model to produce "hallucinations" that are more acceptable on average to human trainers.
Exactly. Don't forget that an important factor in the success of GPT3 was RLHF, which is essentially training the model to produce "hallucinations" that are more acceptable on average to human trainers.