News for hackers. Stuff that matters.
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by MacsHeadroom on Friday June 02, 2023 @06:46AM (#36163480)
Without NVLink different layers are loaded onto individual cards and only one card often has to wait for the other card, slowing down generation. It's still faster than CPU offloading. You can even mix and match GPUs like a 4090 + 3060.

There are no comments on this comment yet.