Re:Salesforce study finds LLM agents flunk CRM and confident...
by skybrian on Monday June 16, 2025 @11:44AM
(#44290680)
Publishing new benchmarks seems useful? If LLM’s improve on this benchmark (and they probably will, like they have on many others) then they’ll need less work on prompting, etc.
[ Parent
| on HN ]
Re:Salesforce study finds LLM agents flunk CRM and confident...
by CityOfThrowaway on Monday June 16, 2025 @12:40PM
(#44291142)
The benchmark is useful, but the conclusion of the write-up is that current generation LLMs can't solve the problem. That's not a valid conclusion to draw. The results here tell us mostly about the skill of the agent-designer, not the capabilities of the model.