> my test prompts in "extra high"
> The model burned through 19200 thinking tokens on this task (which was the hard limit I had set) and didn't finish
I still haven’t caught on to why people keep doing this. Benchmarks too. It feels like a masochistic exercise entirely removed from reality.
Muse Glimmer and Qwen still finish that particular task in their highest reasoning levels without blowing through such a limit.
I will redo my testing at some point at lower effort.
But I would note again that even in extra high thinking mode, it went down a significant rabbit hole to try to make some code work off the basis of a “core” wordpress hook it had actually invented, and nothing I could do would get it to use the correct hook. When I told it about the correct hook what it actually did was engage in a series of tool calls to verify its original understanding, which of course failed to come up with results because the hook it was verifying did not exist. But it continued stubbornly, until it reached a point where it would blankly fail to call tools it said it would call.
I might retest with the Q8 version since it is not that much larger.
FWIW I run Muse Glimmer on “Reasoning strength: low”. It’s perfectly fine; I am not convinced xhigh makes that much difference.
I ultimately simply prefer the way Muse Glimmer works for the turn-by-turn assistant type usage I want; it thinks concisely, writes fairly well too. Struggling a little with tools for it in Pi, though.
> While looping improves the model’s reasoning, it doesn’t enable its weights to store more facts—knowledge is mostly a function of total parameter count. Inkling 2 Mini’s score on AA-Omniscience, a measure of factuality, is in line with other models of the same size. Studying the variable-loop model, we saw that performances on long-context benchmarks improve with more passes, such as GraphWalks BFS and MRCR v2; but knowledge-intensive benchmarks doesn’t improve with more passes, such as AA-Omniscience, SimpleQA Verified, SimpleVQA, and WorldVQA.
> Inkling 2 Mini overcomes the knowledge gap by efficiently using search and tools. It leads open-weights models of similar size on BrowseComp, DeepSearchQA, and MCP Atlas. It is also strong on knowledge work benchmarks that incorporate search and tool use.
My prompt asked it to write a singleton class, single file plugin for WordPress that hooks user login and stores last login time as metadata. It has some code style requirements. The prompt also asked it to make no tool calls so it didn't search and only used its knowledge.
The model burned through 19200 thinking tokens on this task (which was the hard limit I had set) and didn't finish. Its thinking suggests it was going to make a mistake or two.
Both Qwen 3.6 35B and Gemma 4 26B can cope with this task, from memory, in far fewer tokens (even taking into account Qwen's tendency to loop).
Rerunning the same prompt with tool use allowed, it searched with browser use, pretty effectively, and got to a solution, but it still made the mistake the thinking traces suggested it would earlier (wrong hook name). Other than that it wrote good code.
But 20K-ish tokens in, I could then not get it to search to make corrections — it would simply EOT rather than actually run the tools it was saying it would call. Whether this is the model failing or Unsloth's harness and the model disagreeing, I don't know. Quitting and reloading the model didn't help.
It worked through my usual MySQL-5.0-compatible max score per category prompt nicely, though it didn't volunteer the suggestion of adding an index for speed, but it doesn't outperform LFM 2.5-24B-A2B, which is a non-looping model of similar size. It does work through SQL in its thinking traces quite nicely, though.
(I don't have an LFM comparison for the WP thing, I should re-download that and re-run it)
Car wash problem got a correct answer with a surreal caveat (why do they always seem to need to offer a caveat, no matter how weird)
So far, I think I would run Muse Glimmer or Nemotron Bolt over this; Muse Glimmer would output tokens more slowly but get to a result.