News for hackers. Stuff that matters.

A comment on Code Llama, an AI tool for coding. Parent comment

Re:Code Llama, an AI tool for coding by FrozenSynapse on Friday August 25, 2023 @04:40AM (#37259476)
how's the generation speed on CPU?
[ Parent | on HN ]
Re:Code Llama, an AI tool for coding by selfhoster11 on Friday August 25, 2023 @01:40PM (#37264653)
On Ryzen 5600X, 7B and 13B run quite fast. Off the top of my head, pure CPU performance is about 25% slower than with an NVIDIA GPU of some kind. I don't remember the numbers off the top of my head, but the generation speed only starts to get annoying for 33B+ models.
[ Parent | on HN ]