News for hackers. Stuff that matters.

A comment on Announcing GPT-NeoX-20B. Parent comment

Re:Announcing GPT-NeoX-20B by djoldman on Wednesday February 02, 2022 @02:06PM (#30182337)
How much VRAM does it use during inference?
[ Parent | on HN ]
Re:Announcing GPT-NeoX-20B by stellaathena on Wednesday February 02, 2022 @02:32PM (#30182769)
~40 GB with standard optimization. I suspect you can shrink it down more with some work, but it would require significant innovation to cram it into the next largest common chip size (24 GB, unless I’m misremembering)
[ Parent | on HN ]
Re:Announcing GPT-NeoX-20B by komuher on Wednesday February 02, 2022 @04:46PM (#30184503)
Is 40GB already on float16?
[ Parent | on HN ]
Re:Announcing GPT-NeoX-20B by stellaathena on Friday February 04, 2022 @07:25PM (#30215572)
Yes
[ Parent | on HN ]