News for hackers. Stuff that matters.
Re:FTC investigating ChatGPT over potential consumer harm by bodecker on Thursday July 13, 2023 @10:41PM (#36718820)
I assume comments like these, "GPT-4: 8 x 220B experts trained with different data/task distributions and 16-iter inference."

https://twitter.com/soumithchintala/status/16712671501017210... https://archive.li/rfFlW

I'm not sure the most canonical paper on mixture of experts but here's one possible:

https://arxiv.org/pdf/1701.06538.pdf

Re:FTC investigating ChatGPT over potential consumer harm by arthurcolle on Friday July 14, 2023 @02:15AM (#36720094)
I think when ppl refer to MoE they are referring generally to the Google GLaM paper actually