News for hackers. Stuff that matters.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by p1esk on Thursday February 18, 2021 @02:27AM (#26176995)
Yes, you can think of this as a collection of small models, with another model choosing which smaller model to use for each input.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by gwenzek on Thursday February 18, 2021 @07:13AM (#26178940)
Not really, the dispatching between experts happen for every word and every layer, so you can't easily isolate distinct models.