News for hackers. Stuff that matters.

A comment on Llamafile lets you distribute and run LLMs with a single file. Parent comment

Re:Llamafile lets you distribute and run LLMs with a single ... by wazoox on Sunday December 03, 2023 @09:34AM (#38507343)
It's really decent without any GPU. Image analysis is somewhat long, but text prompts are fine. My Ryzen laptop does 2.5 to 4 tokens per second, my Mac pro more like 8.
[ Parent | on HN ]

There are no comments on this comment yet.