News for hackers. Stuff that matters.
Re:Predicting the Order of Upcoming Tokens Improves Language... by NitpickLawyer on Thursday August 28, 2025 @02:45AM (#45049181)
Are any of these methods doable on pre-trained models? Like freeze the model and only train these add-ons? Having to redo the training runs with these optimisations doesn't sound too practical, in the great scheme of things.
Re:Predicting the Order of Upcoming Tokens Improves Language... by impossiblefork on Friday August 29, 2025 @09:28PM (#45071143)
It's obviously practical for the next model you train from scratch. The point of research is obviously not to improve existing commercial products.