Re:A comprehensive guide to running Llama 2 locally
by zakki on Tuesday July 25, 2023 @06:30PM
(#36870580)
Is there any specific settings to make 2x3090 work together?
[ Parent
| on HN ]
Re:A comprehensive guide to running Llama 2 locally
by kwerk on Tuesday July 25, 2023 @11:59PM
(#36872913)
Interested in this too
Re:A comprehensive guide to running Llama 2 locally
by eurekin on Wednesday July 26, 2023 @03:55AM
(#36874407)
Not really? I just got those cards in separate PCI slots and the Exllama_hf handles spreading the load internally. No NVLink bridge in particular. I use the "20,22" memory split so that the display card has some room for the framebuffer to handle display