News for hackers. Stuff that matters.
Re:Running Llama.cpp on AWS Instances by patrakov on Tuesday November 28, 2023 @05:58AM (#38444518)
This is if we follow the recommendation to use a GPU. But if you don't need 200+ tokens per second and are happy with something like 10-15, a CPU-only solution is acceptable and would cost significantly less.

There are no comments on this comment yet.