Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by buildbot on Wednesday February 17, 2021 @07:14PM
(#26173719)
If you read through the paper where they present switch transformers, they train all models to the same performance, so for example t5-large, at 770M parameters[1] is just as good as this model. That’s only 1.54GB
[1] https://huggingface.co/t5-large