Local Qwen isn't a worse Opus, it's a different tool

Re:Local Qwen isn't a worse Opus, it's a different tool by stingraycharles on Thursday June 18, 2026 @08:48AM (#48584529)

Ehr, the SWE bench examples are particularly horrible as those are just publicly available historical PRs. So if the models are trained on GitHub data, it will be included.

So almost by design that particular benchmark is tainted, and benchmarks recall rather than reasoning.

[ Parent | on HN ]

Re:Local Qwen isn't a worse Opus, it's a different tool by willtemperley on Thursday June 18, 2026 @09:10AM (#48584754)

Wow that's worse than I thought, and breaks the number one rule of machine learning: you don't train the model with your test dataset.

[ Parent | on HN ]