Re:Falcon 40B LLM (which beats Llama) now Apache 2.0
by Tepix on Friday June 02, 2023 @07:33AM
(#36163776)
I think there is still lots room for improvement to reduce hardware requirements, such as 3-bit quantization or pruning weights from sparse models.
But if you're willing to spend $1500 on two used RTX 3090, it's the sweet spot in terms of the ability to run large models right now. Everything beyond that is much more expensive.
But if you're willing to spend $1500 on two used RTX 3090, it's the sweet spot in terms of the ability to run large models right now. Everything beyond that is much more expensive.