News for hackers. Stuff that matters.

A comment on Running large language models like ChatGPT on a single GPU. Parent comment

Re:Running large language models like ChatGPT on a single GPU by zeknife on Monday February 20, 2023 @03:29PM (#34872511)
Look up model distillation
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by moffkalast on Monday February 20, 2023 @04:14PM (#34872994)
Ah TIL, so it's actually a thing and with some cleverer approaches too. Then we do have something to look forward to :D
[ Parent | on HN ]