News for hackers. Stuff that matters.

A comment on The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT. Parent comment

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by valine on Sunday April 09, 2023 @09:03PM (#35508065)
I’ve got 2 rtx 4090s on an EATX motherboard. Been using them to run the full 13B un-quantized with a good deal of success. Getting about 20 tokens/s.
[ Parent | on HN ]
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by int_19h on Monday April 10, 2023 @12:13AM (#35509111)
What is your setup for cooling? I don't think I'd want to stick another 4090-size card in mine with just air cooling...
[ Parent | on HN ]
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by valine on Monday April 10, 2023 @12:34AM (#35509212)
I have AIO liquid cooling for both cards. The radiators are annoying though, I might convert it to a custom loop if I ever add a third card.
[ Parent | on HN ]