Re:Falcon 40B LLM (which beats Llama) now Apache 2.0
by airgapstopgap on Wednesday May 31, 2023 @11:38PM
(#36147338)
Multi-Query Attention, used here, should make 40B inference viable on systems where even 33B LLaMA with Multi-Head is basically unusable, so sometimes improvements still come from software optimization (it's no free lunch though).
[ Parent
| on HN ]
There are no comments on this comment yet.