Re:AI agents but they're working in big tech
by henning on Tuesday August 06, 2024 @06:19PM
(#41176014)
- Big tech is very different from open source
- The original SWE-bench paper only consists of solved issues when a big part of Open Source is triage, follow-up, clarification and dealing with crappy issues
- Saying "<Technique> is all you need" when you are increasing your energy usage 30-fold just to fail > 50% of the time is intellectually dishonest
Re:AI agents but they're working in big tech
by alsima on Tuesday August 06, 2024 @06:22PM
(#41176054)
Definitely not saying multi-agents is all you need for SWE-bench haha. I touch on this at the end of the blog post, where I mention jumps in progress require better base models or tooling.
- The original SWE-bench paper only consists of solved issues when a big part of Open Source is triage, follow-up, clarification and dealing with crappy issues
- Saying "<Technique> is all you need" when you are increasing your energy usage 30-fold just to fail > 50% of the time is intellectually dishonest