Introducing DBRX: A New State-of-the-Art Open LLM by Databricks

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by Taek on Wednesday March 27, 2024 @05:21PM (#39844749)

At what parameter count? Its been established that quantization has less of an effect on larger models. By the time you are at 70B quantization to 4 bits basically is negligible

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by K0balt on Wednesday March 27, 2024 @09:10PM (#39846662)

I work mostly with mixtral and mistral 7b these days, but I did work with some 70b models before mistral came out, and I was not impressed with the 4 bit Llama-2 70b.

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by 2099miles on Thursday March 28, 2024 @02:37AM (#39848337)

Source? I’ve seen this anecdotally and heard it, but is there a paper you’re referencing?

[ Parent | on HN ]