News for hackers. Stuff that matters.
Re:GPT-4 Details Leaked by pas on Tuesday July 11, 2023 @09:09AM (#36680452)
Interestingly Google was using ~2000 experts back in the first Trasnformer architecture (if I understand correctly) https://www.youtube.com/watch?v=9P_VAMyb-7k&t=6m42s [sparsely-gated mixture of experts layer]

There are no comments on this comment yet.