Qwen2.5-Coder-32B is an LLM that can code well that runs on my Mac

Re:Qwen2.5-Coder-32B is an LLM that can code well that runs ... by JimDabell on Wednesday November 13, 2024 @06:17AM (#42125030)

> The issue with some recent models is that they're basically overfitting on public evals… You really want to be testing stuff that isn't overfit to death… I'd be shocked if this model held up in the comprehensive private evals.

From the announcement:

> we selected the latest 4 months of LiveCodeBench (2024.07 - 2024.11) questions as the evaluation, which are the latest published questions that could not have leaked into the training set, reflecting the model’s OOD capabilities.

— https://qwenlm.github.io/blog/qwen2.5-coder-family/

[ Parent | on HN ]

Re:Qwen2.5-Coder-32B is an LLM that can code well that runs ... by tucnak on Wednesday November 13, 2024 @07:07AM (#42125337)

They say a lot of things, like that their base models weren't instruction-tuned, however people have confirmed that it's impossible to find instruction that it wouldn't follow, and the output would indicate that exactly. The labs absolutely love incorporating public evals in their training; of course, they're not going to admit that.

[ Parent | on HN ]

Re:Qwen2.5-Coder-32B is an LLM that can code well that runs ... by throwdbaaway on Friday November 15, 2024 @09:15AM (#42147131)

I am running the base model of Qwen2.5-Coder-32B with llama.cpp. It can only do completion, it can't chat. Where did you get that information from?

[ Parent | on HN ]