Re:Falcon 40B LLM (which beats Llama) now Apache 2.0
by senttoschool on Thursday June 01, 2023 @09:50AM
(#36151005)
>From what I understand, even to run locally you/your team needs to be able to afford a machine with a 4090. These are super expensive in some countries.
That's because we're only half a year into LLMs becoming mainstream. Give it 3-4 years. The advancements in bringing down model size, optimizations, and newer GPUs, SoCs from Nvidia, AMD, Apple, Intel, Qualcomm, etc will make it so that top LLMs will run on a highend laptop/desktop.
That's because we're only half a year into LLMs becoming mainstream. Give it 3-4 years. The advancements in bringing down model size, optimizations, and newer GPUs, SoCs from Nvidia, AMD, Apple, Intel, Qualcomm, etc will make it so that top LLMs will run on a highend laptop/desktop.