News for hackers. Stuff that matters.

A comment on Ask HN: Cheapest hardware to run Llama 2 70B.

Re:Ask HN: Cheapest hardware to run Llama 2 70B by tuxpenguine on Wednesday August 09, 2023 @04:45PM (#37068132)
I don't think it is the cheapest, but the tiny box is an option:

https://tinygrad.org/

[ Parent | on HN ]
Re:Ask HN: Cheapest hardware to run Llama 2 70B by Tepix on Thursday August 10, 2023 @11:57AM (#37077846)
Tiny box is for running it at fp16, most other comments here talk about running it with 4bit quantization
[ Parent | on HN ]