News for hackers. Stuff that matters.

A comment on Running Llama.cpp on AWS Instances.

Re:Running Llama.cpp on AWS Instances by ilaksh on Tuesday November 28, 2023 @04:06AM (#38443845)
I am more interested on running llama.cpp on CPU-only VPSs/EC2. Although it is probably too slow.
[ Parent | on HN ]
Re:Running Llama.cpp on AWS Instances by rini17 on Tuesday November 28, 2023 @05:10AM (#38444244)
13b versions of models are running on 8-core CPU fast enough to have a fluid conversation.
[ Parent | on HN ]
Re:Running Llama.cpp on AWS Instances by borissk on Tuesday November 28, 2023 @11:35AM (#38447591)
What can I run on nVidia RTX 4060 Ti with 16GB RAM?
[ Parent | on HN ]
Re:Running Llama.cpp on AWS Instances by rini17 on Tuesday November 28, 2023 @05:03PM (#38451987)
Best to try yourself, llama.cpp is refreshingly easy to build.
[ Parent | on HN ]