chaosadm
Re:Show HN: A business SIM where humans beat GPT-5 by 9.8 X
on Wednesday November 19, 2025 @05:33PM (#45986235)
The game environment looks pretty neat. Not surprised to see LLMs struggling but with a benchmark to focus new techniques on, I am excited how some of the new solutions trying to top the leaderboard would do.
Re:Show HN: A business SIM where humans beat GPT-5 by 9.8 X
on Wednesday November 19, 2025 @05:26PM (#45986163)
So there are a couple of papers that try to use LLMs for UI-based enterprise task benchmarking like WorkArena++(ServiceNow) where the agent has to solve a couple of relatively simple enterprise tasks (like creating incident tickets based on some criteria that has to be determined by the agent etc). This benchmark in particular had quite low accuracy numbers especially on the more composite tasks. Curious about the OpenAI Atlas thing too.