[0] Not even here: https://playground.microsoft.ai/
Qwen-3.6-27b is closer to Claude Opus 4.7 than it is to Haiku 4.5 in a lot of benchmarks - and it's way smaller than Microsoft's new model.
Sure, it competes with Haiku, but it shows how far Microsoft is behind lots of other small models that are available.
Instead, they only do cherry-picked comparisons against Anthropic's small models, and not the full spectrum of competitors.
Without evidence to the contrary, I'll interpret this as just what happens when you're late to the party and insist on doing everything from scratch.
Maybe coaxing reasoning behavior out of their base model without kickstarting it by distilling from existing models provided them with valuable experience that will help improve their future models, or maybe it was an unnecessary waste of time.
I don’t think that’s right, this flash model is 5B active params. Qwen3.6-35B-A3B is 3B so 40% smaller.
Yeah, not a 5B param model as the earlier title implied!
https://news.ycombinator.com/formatdoc
crimes ↑
│
10.0 ┤ ● Airport burger
│ ╭──────────────╮
8.0 ┤ │ theft arc │
│ ╰──────────────╯
6.0 ┤ ● Five Guys
│
4.0 ┤ ● Food truck burger
│
2.0 ┤ ● McBurger
│
0.0 ┤ ● Homemade burger
│
└───────┬─────────┬─────────┬─────────┬─────────→ price
$2 $8 $14 $22 $38
┌────────────────────┬────────┬──────────────┬────────────────────┐
│ burger │ price │ crime index │ expected behavior │
├────────────────────┼────────┼──────────────┼────────────────────┤
│ Homemade burger │ $2 │ 0.0 │ law-abiding citizen│
│ McBurger │ $6 │ 1.4 │ steals extra napkin│
│ Food truck burger │ $11 │ 3.1 │ lies about hunger │
│ Five Guys │ $18 │ 6.2 │ financial crime │
│ Airport burger │ $34 │ 9.7 │ enters villain arc │
└────────────────────┴────────┴──────────────┴────────────────────┘
conclusion: burger inflation is a gateway condimentThe next question is where did the "ASCII-art" graph and table come from? Are there sites to generate these?
Just built a tool for that: https://krysoph.github.io/UnicodeData/
It is a single html file with no dependencies, it takes json data and turns into unicode charts.
If I use your tool and "Copy HN-ready" and paste here then it works, but oddly if I then edit the post the formatting is lost.
Also, if I just manually post, starting with a blank line, followed by a couple of lines starting with two spaces (e.g. " aaa", " bbb"), then I'm not getting the <pre> code formatting. Any idea what I might be doing wrong?
I think HN strips starting blank lines,
but it may work when done after a line with some char like: .
edit test: 0 ┤ ╭───╮ Homer made burger
│ │ ● │ ╭───╮ McBurger
2 ┤ ╰───╯ │ ○ │ ╭─────╮
│ ╰───╯ │ * │ Food truck burger
4 ┤ ╰─────╯
│
│ ╭─────╮
6 ┤ │ ■ │ this was edited btw
│ ╰─────╯ ╭─────╮
8 ┤ Hairport burger │ ▲ │
│ ╰─────╯
10 └┬────────────┬───────────┬────────────┬───────────┬────────────┬
$0 $8 $16 $24 $32 $40
Works fine for me, even after multiple edits the formatting stays the same ¯\(ツ)/¯ aa
bbOn benchmarks: in the same VS Code harness, MAI-Code-1-Flash scored 51.2% on SWE-bench Pro vs. Haiku's 35.2% which we see as a pretty big leap. But going forward, we'll include additional models in our benchmarks, including models like Qwen 3.6 and Gemma 4.
Even if it can't fully pass much, there are so many tests against most of the scenarios that you can get a fairly rich report beyond the pass@1 stat. See e.g. this DeepSWE report against the Minimax M3 model: https://entrpi.github.io/misc/deep-swe-minimax-m3/
Sorry Trump Administration, but while the US has been downloading more ram by throwing data centers at everything and burning up everyone's power and water, China has come out with what's effectively a prototype edge compute capable AI model - regardless of how they built it. And arguably I can tokenmaxx on it just fine at around 30-40 tokens/sec.
And also, ASICs are on the way. Imagine one of those with a heavy hitting model (MoE or otherwise, Qwen or otherwise) installed in a PCIe slot at 10k+ tokens/sec and 75 watts max (maximum wattage deliverable by the PCIe slot alone) for $300-400 USD each.
https://taalas.com/the-path-to-ubiquitous-ai/
ASIC demo here: https://chatjimmy.ai/
Sorry/not sorry to rip this whole thing to shreds. But I'm sick and tired of these inefficient LLMs being produced that seemingly can only be offered by subscription from a data center, when I'm running a full AI stack right now (model and all) on my computer at home on a 750 watt max power supply. Microsoft really needs to get with the picture here and compete more with Qwen instead of just the US/EU entities.
Sincerely, your neighbor down in Tacoma. https://www.youtube.com/watch?v=V9jlo4Ht2YA&t=229s
I was implementing a re-print functionality in my warehouse management system.
It took Opus 4.8 high 24m1s and 87k tokens. Took Haiku 6m30s and 41k tokens.
After that time I had to provide (minor) adjustments to both. But Haiku allowed me to iterate faster. Code quality for that somewhat trivial use case was similar.
Actually, I would even say that Opus provided a sub par solution: instead of fixing an issue where carrier label pdf wasn't saved as the state machine progressed to the latest step, it went through a much complex solution of re-generating those by scratch. Which is also wrong, as it was de-facto booking the carriers twice for the same order.
Haiku simply added another field on the terminal state that carried the already generated urls.
I don't think it's a good idea to default to highest effort/bigger model without taking into account the time it takes and the task complexity.
Imho we should experiment rather than assume that what the rest of the community does to be the best practice.
Yesterday Codex was making a big issue out of a new module that was upgraded in our cluster and because of which the same SSH key would be "regenerated" by Terraform. No big deal, it just truncates a newline at the end of the SSH key and it works all the same. But not being aware that this, as an example, is unimportant can cost a lot more time than using the big models saves.
[1] https://srinathh.medium.com/mid-size-local-models-are-now-co...
Deepseek seems to seek first to understand before going off.
I think there's possibly value to try fine tuning Qwen 3.5 on my OpenClaw turns log to see if performance improves. The one recent model I haven't tested yet is Nemotron 3 Super which I might bench soon.
Performance doesn't seem that good:
- MAI-Code-1-Flash (137B-A5B) = 51% on SWE-bench pro
- Qwen3.6-35B-A3B = 49.5% on SWE-bench pro (https://huggingface.co/Qwen/Qwen3.6-35B-A3B)
They benchmark against Claude Haiku but Haiku is not good, it's worse than tiny open models you can run locally or via API at 10% the cost.