Do my comments get tiresome, though? None have been downvoted to hell, since they are all adding to discussion. If you think so feel free to email me, I don't want to be annoying.
Genuinely, just excited about what we're building and think it adds information about these models that are missing.
BTW check out my totally real project that helps you detect native advertising on your favorite platforms:
Web: 87 (12.2%) Systems and protocols: 78 (10.9%) Software, unspecified type: 74 (10.4%) Backend: 68 (9.5%) Writing and documentation: 55 (7.7%) Research and advice: 47 (6.6%) Unknown: 42 (5.9%) Mobile: 40 (5.6%) Operations and automation: 36 (5.0%) Developer tooling: 35 (4.9%) Desktop: 28 (3.9%) Media and design: 27 (3.8%) Data analysis: 25 (3.5%) Device and system support: 22 (3.1%) Math and formal methods: 18 (2.5%) Games and simulations: 15 (2.1%) Infrastructure and DevOps: 9 (1.3%) AI and machine learning: 6 (0.8%) Other: 1 (0.1%)
We've been trying to capture real SWE data by building a battle system on top of OpenCode. In our data, Sol 6.1 is seriously underperforming, I'm talking worse than DeepSeek 4.1 Flash, at 10x the cost. (We allow votes before both agents are done, contrary to typical Arena methodology, because we think a fast good enough answer is better than a slow one. https://agentduel.com/leaderboard).
OpenAI performance on our system has been bad enough I switched to Anthropic models for my daily work.