News for hackers. Stuff that matters.
Re:Pile-T5 by jakderrida on Wednesday April 17, 2024 @06:31PM (#40070766)
I've downloaded the data for every model on Huggingface and, if nothing else, flan-t5-xxl is the largest encoder (discriminant) model in terms of total file sizes. To be clear T5 is an encoder-decoder model. But the shift in focus to decoder-only has made it so encoder and encoder-decoder models have been, for lack of a better word, neglected, in terms of training focus.

There are no comments on this comment yet.