News for hackers. Stuff that matters.
Re:A Replacement for Bert by janalsncm on Thursday December 19, 2024 @04:29PM (#42465870)
On your second point, most modern LLMs are decoder only. And as for why adding a classification head isn’t optimal, the decoders you’re referring to have 10x the parameters, and aren’t trained on encoder-type tasks like MLM. So there’s no advantage on any dimension really.

There are no comments on this comment yet.