News for hackers. Stuff that matters.
Re:New #1 open-source AI Agent on SWE-bench Verified by candiddevmike on Thursday May 22, 2025 @07:43AM (#44061059)
SWE bench scores, like a lot of other metrics for LLMs, are pretty divorced from reality IMO. It's a lot like only learning to pass tests vs actual understanding.

Once GenAI companies stop hiring SWEs, I'll believe the doomers.

Re:New #1 open-source AI Agent on SWE-bench Verified by harshitaneja on Thursday May 22, 2025 @08:04AM (#44061209)
I help hire for a few clients as well as for my own small organization. We are already seeing impact of these tools on our hiring. For the same responsibilities and tasks we are already requiring lesser resources. For clients with less complex problems we are able to manage similar work with 60% of the resources planned. And that's when most of our work is mathematical modelling, heuristics, constraint programming and such. However, I don't foresee at least for the next few years we would ever get to a scenario where we don't hire developers. Given that most hiring has shifted to only senior developers.
Re:New #1 open-source AI Agent on SWE-bench Verified by dingnuts on Thursday May 22, 2025 @10:48AM (#44062653)
being able to do more things with fewer resources (which lowers costs) always increases demand enough to make up for the reduction of labor caused by the automation

Analogy: when the chainsaw was invented, we didn't stop having lumberjacks, they just learned to use chainsaws

Re:New #1 open-source AI Agent on SWE-bench Verified by MukundMohanK on Thursday May 22, 2025 @08:04AM (#44061208)
Reality is here whether we like it or not - https://fred.stlouisfed.org/graph/?g=1DEP0
Re:New #1 open-source AI Agent on SWE-bench Verified by hackeman300 on Thursday May 22, 2025 @09:15AM (#44061753)
Surely there are no other macroeconomic factors that could have played a role in this decline too