Introducing DBRX: A New State-of-the-Art Open LLM by Databricks

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by briandw on Wednesday March 27, 2024 @12:50PM (#39841550)

Worse than the chart crime of truncating the y axis is putting LLaMa2's Human Eval scores on there and not comparing it to Code Llama Instruct 70b. DBRX still beats Code Llama Instruct's 67.8 but not by that much.

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by jjgo on Wednesday March 27, 2024 @02:04PM (#39842531)

> "On HumanEval, DBRX Instruct even surpasses CodeLLaMA-70B Instruct, a model built explicitly for programming, despite the fact that DBRX Instruct is designed for general-purpose use (70.1% vs. 67.8% on HumanEval as reported by Meta in the CodeLLaMA blog)."

To be fair, they do compare to it in the main body of the blog. It's just probably misleading to compare to CodeLLaMA on non coding benchmarks.

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by tartrate on Wednesday March 27, 2024 @04:00PM (#39843872)

Which non-coding benchmark?

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by panarky on Wednesday March 27, 2024 @09:21PM (#39846736)

> chart crime of truncating the y axis

If you chart the temperature of the ocean do you keep the y-axis anchored at zero Kelvin?

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by d-z-m on Wednesday March 27, 2024 @11:05PM (#39847334)

If you chart the temperature of the ocean are you measuring it in Kelvin?

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by panarky on Thursday March 28, 2024 @03:26PM (#39856266)

Apparently, if you want to avoid "chart crime" when you chart temperatures, then it's deceptive if you don't start at absolute zero.

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by evrial on Thursday March 28, 2024 @07:40PM (#39858810)

When was the temperature on Earth at absolute zero?

[ Parent | on HN ]

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by panarky on Friday March 29, 2024 @01:53PM (#39866994)

My point exactly.

In a chart of world gross domestic product for the last 12 months, when was it at zero?

In a chart of ocean salinity, when was it at absolute zero?

Is it inherently deceptive to use a y-axis that doesn't begin at zero?

[ Parent | on HN ]