Re:Localllm lets you develop gen AI apps on local CPUs
by depingus on Thursday February 08, 2024 @11:07AM
(#39303600)
I wasn't really timing it, but it felt fine, especially since it prints out while generating. The 13B model on the 5600X is about 10 seconds. Tho once the conversation gets too long (about 10 replies) it will go sideways. I gotta play with the default settings. IIRC the 7B on the Fx8350 was closer to 25 seconds per response.
[ Parent
| on HN ]
There are no comments on this comment yet.