Re:A comprehensive guide to running Llama 2 locally
by eurekin on Wednesday July 26, 2023 @03:55AM
(#36874407)
Not really? I just got those cards in separate PCI slots and the Exllama_hf handles spreading the load internally. No NVLink bridge in particular. I use the "20,22" memory split so that the display card has some room for the framebuffer to handle display
[ Parent
| on HN ]