News for hackers. Stuff that matters.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by buildbot on Wednesday February 17, 2021 @07:14PM (#26173719)
If you read through the paper where they present switch transformers, they train all models to the same performance, so for example t5-large, at 770M parameters[1] is just as good as this model. That’s only 1.54GB

[1] https://huggingface.co/t5-large

There are no comments on this comment yet.