News for hackers. Stuff that matters.

A comment on Tinygrad: Hacked 4090 driver to enable P2P. Parent comment

Re:Tinygrad: Hacked 4090 driver to enable P2P by wtallis on Friday April 12, 2024 @02:41PM (#40016274)
Do people do training on systems this small, or just inference? I could see maybe doing a little bit of fine-tuning, but certainly not from-scratch training.
[ Parent | on HN ]
Re:Tinygrad: Hacked 4090 driver to enable P2P by redox99 on Friday April 12, 2024 @05:01PM (#40017595)
If you mean train llama from scratch, you aren't going to train it on any single box.

But even with a single 3090 you can do quite a lot with LLMs (through QLoRA and similar).

[ Parent | on HN ]
Re:Tinygrad: Hacked 4090 driver to enable P2P by thangngoc89 on Saturday April 13, 2024 @12:33AM (#40020519)
Yep. Price/performance of multiple 4090s system are way better than the professional cards (Axxx). Also deep learning outside of LLM has many different usage.
[ Parent | on HN ]