News for hackers. Stuff that matters.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by lucubratory on Friday August 04, 2023 @07:12PM (#37007065)
It was completely unsubstantiated, based on rumours from a blog, but everyone repeated it as fact.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by whimsicalism on Friday August 04, 2023 @07:53PM (#37007398)
I think it is pretty compelling that almost all of the people doing research into switch transformers at Google were hired into OAI. I am not sure if that is ouboicly reported but once Ghotz leaked those details about the models, I went to check where the authirs of those papers are now and.... yep