News for hackers. Stuff that matters.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by goodside on Friday August 04, 2023 @07:34PM (#37007240)
You do, because it’s not just more training it’s PPO updates instead of MLE. It’s no longer trying to estimate the token distribution of the training corpus, it’s trying to shift logprobs into tokens that maximize expected reward from the RM. The GPT-4 technical report has a figure showing that logprobs become less well calibrated as confidence scores in the RLHF vs pre-train model.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by refulgentis on Friday August 04, 2023 @09:47PM (#37008163)
Fascinating, ty