Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Ask HN: Cheapest hardware to run Llama 2 70B. Parent comment
https://replicate.com/blog/run-llama-locally
I found that guide here on hn.
I run it cpu only with 16 threads but yeah perf is good enough.
BTw my 6gb figure is me.measuring from htop so llama2 is likely less.
https://replicate.com/blog/run-llama-locally
I found that guide here on hn.
I run it cpu only with 16 threads but yeah perf is good enough.
BTw my 6gb figure is me.measuring from htop so llama2 is likely less.