News for hackers. Stuff that matters.
Re:We tasked Opus 4.6 using agent teams to build a C Compiler by dcre on Friday February 06, 2026 @04:04PM (#46918109)
That is not what No True Scotsman is. I’m pointing out a bad argument with weak evidence.
Re:We tasked Opus 4.6 using agent teams to build a C Compiler by thesz on Friday February 06, 2026 @04:34PM (#46918452)

  >>> It is a very contrived under-specified prompt.
No True Prompt can be such contrived and underspecified.

The article about degradation is a case study (single prompt), weakest of the studies in hierarchy of knowledge. Case studies are basis for further, more rigorous studies. And author took the time to test his assumptions and presented quite clear evidence that such degradation might be present and that we should investigate.

Re:We tasked Opus 4.6 using agent teams to build a C Compiler by dcre on Friday February 06, 2026 @10:18PM (#46920987)
We have investigated. Millions of people are investigating all the time and finding that the coding capacity has improved dramatically over that time. A variety of very different benchmarks say the same. This one random guy’s stupid prompt says otherwise. Come on.
Re:We tasked Opus 4.6 using agent teams to build a C Compiler by thesz on Saturday February 07, 2026 @02:54AM (#46922126)
As far as I remember, article stated that he found same problematic behavior for many prompts, issued by him and his colleagues. The "stupid prompt" in article is for demonstration purposes.
Re:We tasked Opus 4.6 using agent teams to build a C Compiler by dcre on Monday February 09, 2026 @09:15AM (#46945430)
But that’s not an argument, that’s just assertion, and it’s directly contradicted by all the more rigorous attempts to do the same thing through benchmarks (public and private).