Re:Bad numbers in the “gzip beats BERT” paper?
by tensor on Monday July 17, 2023 @12:34PM
(#36760316)
FastText isn't a LLM, it's a token embedding model with a simple classifier on top.
[ Parent
| on HN ]
Re:Bad numbers in the “gzip beats BERT” paper?
by marcinzm on Monday July 17, 2023 @01:49PM
(#36761512)
Sure but it's existence means the statement is really "gzip approach not better than dnn models, and doesn't compete or be cheaper to run than previous models like FastText." That's not a very meaningful value statement for the approach (although why gzip is even half-decent might be a very interesting research question).