News for hackers. Stuff that matters.
Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by ikurei on Thursday June 04, 2026 @04:38AM (#48395879)
Qwen 3.7 Max: > During my local testing before the full eval harness it was the only non-GPT model that was able to complete the task, was not able to reproduce in the longer runs.

Doesn't that sound like may be the harness was the problem?

Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by jc4p on Thursday June 04, 2026 @11:25AM (#48400086)
I was using the same harness for each run, the difference is from when I was running the harness locally on my machine before I pushed up the full runs.