News for hackers. Stuff that matters.

A comment on I built a vulnerable app and spent $1,500 seeing if LLMs could hack it. Parent comment

Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by baldai on Thursday June 04, 2026 @06:07AM (#48396456)
They are not even close in capabilities. Only nenchmark I ever seen that captures their difference is DeepSWE. They are worse by factor of 3.
[ Parent | on HN ]
Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by Cakez0r on Thursday June 04, 2026 @07:19AM (#48397051)
Here are 3 benchmarks showing the comparable scores I was talking about

https://openrouter.ai/rankings https://arena.ai/leaderboard/text/coding https://artificialanalysis.ai/

[ Parent | on HN ]
Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by jona-f on Thursday June 04, 2026 @12:31PM (#48401008)
Wait, the only benchmark you found? It looks like you never heard of confirmation bias before. https://en.wikipedia.org/wiki/Confirmation_bias
[ Parent | on HN ]