Running Llama.cpp on AWS Instances

Re:Running Llama.cpp on AWS Instances by ionwake on Tuesday November 28, 2023 @05:24AM (#38444312)

SO roughyl how much does this instance cost a day? Like $30? Im kind of confused why it wasnt mentioned, but hey maybe poeple arent as cheap as me. Cool project tho.

[ Parent | on HN ]

Re:Running Llama.cpp on AWS Instances by michaelt on Tuesday November 28, 2023 @05:29AM (#38444341)

The pricing is shown in the first image on the page: $0.526 per hour.

[ Parent | on HN ]

Re:Running Llama.cpp on AWS Instances by patrakov on Tuesday November 28, 2023 @05:58AM (#38444518)

This is if we follow the recommendation to use a GPU. But if you don't need 200+ tokens per second and are happy with something like 10-15, a CPU-only solution is acceptable and would cost significantly less.

[ Parent | on HN ]