News for hackers. Stuff that matters.
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by airgapstopgap on Wednesday May 31, 2023 @11:38PM (#36147338)
Multi-Query Attention, used here, should make 40B inference viable on systems where even 33B LLaMA with Multi-Head is basically unusable, so sometimes improvements still come from software optimization (it's no free lunch though).

There are no comments on this comment yet.