News for hackers. Stuff that matters.

A comment on OpenAI ChatGPT: Optimizing Language Models for Dialogue. Parent comment

Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by zaptrem on Wednesday November 30, 2022 @10:37PM (#33812040)
I'm able to run a 22b parameter GPT-Neo model on my 24gb 3090 and can fit a 30b parameter OPT model when combining my 3090 and 12gb 3080
[ Parent | on HN ]
Re:OpenAI ChatGPT: Optimizing Language Models for Dialogue by mdda on Thursday December 01, 2022 @06:32AM (#33814972)
Could you point to any resources online about how to do this? e.g. is this using 8-bit quantisation?
[ Parent | on HN ]