Re:I built a vulnerable app and spent $1,500 seeing if LLMs ...
by ikurei on Thursday June 04, 2026 @04:38AM
(#48395879)
Qwen 3.7 Max:
> During my local testing before the full eval harness it was the only non-GPT model that was able to complete the task, was not able to reproduce in the longer runs.
Doesn't that sound like may be the harness was the problem?
Re:I built a vulnerable app and spent $1,500 seeing if LLMs ...
by jc4p on Thursday June 04, 2026 @11:25AM
(#48400086)
I was using the same harness for each run, the difference is from when I was running the harness locally on my machine before I pushed up the full runs.
Doesn't that sound like may be the harness was the problem?