News for hackers. Stuff that matters.
Re:Anthropic's original take home assignment open sourced by saagarjha on Wednesday January 21, 2026 @04:21AM (#46703099)
Why do you assume it’s cheating?
Re:Anthropic's original take home assignment open sourced by measurablefunc on Wednesday January 21, 2026 @05:55AM (#46703821)
Because it's a well know failure mode of neural networks & scalar valued optimization problems in general: https://www.nature.com/articles/s42256-020-00257-z
Re:Anthropic's original take home assignment open sourced by saagarjha on Wednesday January 21, 2026 @06:41AM (#46704285)
Again, you can just read the code
Re:Anthropic's original take home assignment open sourced by measurablefunc on Wednesday January 21, 2026 @11:46AM (#46708131)
You're missing the point. There is no evidence to support their claims which means they are more than likely leaking the memory into the LLM prompt & it is cheating by simply loading constants into memory instead of computing anything. This is why formal specifications are used to constrain optimization. Without proof that the code is equivalent you might as well just load constants into memory & claim victory.
Re:Anthropic's original take home assignment open sourced by fc417fc802 on Wednesday January 21, 2026 @12:23PM (#46708631)
> There is no evidence to support their claims

Do you make a habit of not presuming even basic competence? You believe that Anthropic left the task running for hours, got a score back, and never bothered to examine the solution? Not even out of curiosity?

Also if it was cheating you'd expect the final score to be unbelievably low. Unless you also suppose that the LLM actively attempted to deceive the human reviewers by adding extra code to burn (approximately the correct number of) cycles.

Re:Anthropic's original take home assignment open sourced by measurablefunc on Wednesday January 21, 2026 @01:48PM (#46709778)
This has nothing to do w/ me & consistently making it a personal problem instead of addressing the claims is a common tactic for people who do not know what it means to present evidence for their claims. Anthropic has not provided the necessary evidence for me to conclude that their LLM is not cheating. I have no opinion on their competence b/c that is not what is at issue. They could be incompetent & not notice that their LLM is cheating at their take home exam but I don't care about that.
Re:Anthropic's original take home assignment open sourced by fc417fc802 on Wednesday January 21, 2026 @02:00PM (#46709934)
You are implying that you believe them to be incompetent since otherwise you would not expect evidence in this instance. They also haven't provided independent verification of their claims - do you suspect them of lying as well?

How do you explain the specific score that was achieved if as you suggest the LLM simply copied the answer directly?

Re:Anthropic's original take home assignment open sourced by measurablefunc on Wednesday January 21, 2026 @02:47PM (#46710600)
Either they have proof that their LLM is not cheating or they don't. The linked post does not provide evidence that the LLM is not cheating. I don't have to explain anything on my end b/c my claim is very simple & easily refuted w/ the proper evidence.
Re:Anthropic's original take home assignment open sourced by red75prime on Wednesday January 21, 2026 @06:32AM (#46704187)
And? Anthropic is not aware of this 2020 paper? The problem is not solvable?
Re:Anthropic's original take home assignment open sourced by measurablefunc on Wednesday January 21, 2026 @11:53AM (#46708223)
Why are you asking me? Email & ask Anthropic.
Re:Anthropic's original take home assignment open sourced by red75prime on Wednesday January 21, 2026 @07:28PM (#46713648)
Obviously, because you use this old paper as an argument.
Re:Anthropic's original take home assignment open sourced by measurablefunc on Wednesday January 21, 2026 @07:35PM (#46713712)
I don't have any insider information on what they know or don't know so you're welcome to keep asking nonsensical questions but eventually I'll stop answering.