News for hackers. Stuff that matters.

A comment on Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML. Parent comment

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by Tostino on Monday June 26, 2023 @11:18AM (#36480803)
Falcon is also slow for inference compared to LLaMA models of similar size. Speed can turn into a quality all its own when you break up problems into small pieces and use the LLM to iterate.
[ Parent | on HN ]
Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by brucethemoose2 on Monday June 26, 2023 @11:52AM (#36481394)
Thats because its unoptimized, right? My impression is that the Falcon GPTQ (and 8 bit bnb inference?) code is just immature.
[ Parent | on HN ]