Re:A Replacement for Bert
by janalsncm on Thursday December 19, 2024 @04:29PM
(#42465870)
On your second point, most modern LLMs are decoder only. And as for why adding a classification head isn’t optimal, the decoders you’re referring to have 10x the parameters, and aren’t trained on encoder-type tasks like MLM. So there’s no advantage on any dimension really.
[ Parent
| on HN ]
There are no comments on this comment yet.