Re:Falcon 40B LLM (which beats Llama) now Apache 2.0
by dzhiurgis on Wednesday May 31, 2023 @10:42PM
(#36147030)
> HF requires you to use their commercial offerings
Some of which are quite affordable ($80 per month). Larger ones can be like 2000 a month which is still ok to prototyping phase. You're basically paying for aws/gcp infrastructure.
I quite liked the UX of it, very intuitive. My trouble was finding a model that executes out-of-the-box tho. All of the GPT ones crash on startup.
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0
by YetAnotherNick on Friday June 02, 2023 @05:55AM
(#36163173)
I spent some time today exploring HuggingFace's Inference API but if the model is sufficiently large (> 10gb), HF requires you to use their commercial offerings.