I'm making LLMs work on small and old devices

Re:I'm making LLMs work on small and old devices by rybosome on Thursday November 12, 2026 @07:10PM (#50476340)

I find this compelling “merely” as an engineering challenge, but I’m curious if you’ve observed an actual practical benefit?

By that I mean, have you found a case where a fine-tuned, small model is outperformed by stuffing a heavily tuned larger model in its place?

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by mrashad74 on Friday November 13, 2026 @06:42AM (#50482121)

I had this case in fine tuned small models (up to 27B) in finance and legal, over bigger models (cost-wise, on-prem for a regulated institution, and better perofrmance because we owned the data pipeline)

[ Parent | on HN ]