Re:Ask HN: What's something interesting you learned from tra...
by linolevan on Tuesday January 27, 2026 @02:11PM
(#46784797)
For tiny models, the SFT data mixture is unbelievably critical to usability. They are unable to generalize in almost any way. If you don't have multi-turn conversations, they will not be able to do multi-turn conversations. If you have multi-turn conversations which are just chatting, and then single turn conversations for math, it will be unable to do math in a multi-turn setting. This is much less true for bigger models.
[ Parent
| on HN ]
There are no comments on this comment yet.