News for hackers. Stuff that matters.

A comment on Introducing DBRX: A New State-of-the-Art Open LLM by Databricks. Parent comment

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by jjgo on Wednesday March 27, 2024 @02:04PM (#39842531)
> "On HumanEval, DBRX Instruct even surpasses CodeLLaMA-70B Instruct, a model built explicitly for programming, despite the fact that DBRX Instruct is designed for general-purpose use (70.1% vs. 67.8% on HumanEval as reported by Meta in the CodeLLaMA blog)."

To be fair, they do compare to it in the main body of the blog. It's just probably misleading to compare to CodeLLaMA on non coding benchmarks.

[ Parent | on HN ]
Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by tartrate on Wednesday March 27, 2024 @04:00PM (#39843872)
Which non-coding benchmark?
[ Parent | on HN ]