News for hackers. Stuff that matters.

A comment on The Coming of Local LLMs. Parent comment

Re:The Coming of Local LLMs by anonzzzies on Tuesday April 11, 2023 @01:46PM (#35528542)
Can you calculate that for me on a napkin? Every calculation I make for training, but certainly for inference, makes me break even after well under a year and then it’s vastly less if I buy the hardware myself.
[ Parent | on HN ]
Re:The Coming of Local LLMs by whimsicalism on Tuesday April 11, 2023 @01:50PM (#35528603)
You can infer on your local machine.

I doubt it will take you a year to train a model.

Vicuna-13B cost $300 to train/fine-tune [0]. They trained on an A100 which costs $10k [1].

[0]: https://vicuna.lmsys.org/ [1]: https://github.com/lm-sys/FastChat/blob/main/scripts/train-v...

[ Parent | on HN ]