News for hackers. Stuff that matters.

A comment on What Is ChatGPT Doing and Why Does It Work?. Parent comment

Re:What Is ChatGPT Doing and Why Does It Work? by sigmoid10 on Tuesday February 14, 2023 @08:21PM (#34798798)
Knowledge distillation for transformers is already a thing and it is still actively researched since the potential benefits of not having to run these gigantic models are enormous.
[ Parent | on HN ]
Re:What Is ChatGPT Doing and Why Does It Work? by lossolo on Tuesday February 14, 2023 @09:50PM (#34799404)
Indeed but all the papers I've read about this were only for simple transformers, for now at least.
[ Parent | on HN ]