Re:BigCode Project Releases StarCoder: A 15B Code LLM
by tmaly on Thursday May 04, 2023 @05:30PM
(#35822353)
I might have missed it, what are the minimum hardware requirements needed to run this?
[ Parent
| on HN ]
Re:BigCode Project Releases StarCoder: A 15B Code LLM
by enum on Thursday May 04, 2023 @05:44PM
(#35822503)
You can just about load it on a 32GB GPU in 16bit mode.
Quantized versions here:
https://huggingface.co/mayank31398/starcoder-GPTQ
they will be benchmarked on humaneval and released soon—maybe tomorrow?