News for hackers. Stuff that matters.
Re:Alice in Wonderland: Simple Tasks Showing Complete Reason... by oidar on Wednesday June 05, 2024 @10:39AM (#40585367)
Of course it's going to give an incorrect answer with that prompt. If the instruction fine tuning is neutered like this prompt, it's going to roll over to the foundation model and offer a completion - probably more influenced by the seed than the prompting text. Bad study.

Edit - I just skimmed the paper - they do use other more appropriate prompt types for reasoning. My initial response was based on the assumption that all prompts used that script prompt quoted in the parent. I retract my "bad paper" comment.

Re:Alice in Wonderland: Simple Tasks Showing Complete Reason... by YeGoblynQueenne on Wednesday June 05, 2024 @04:23PM (#40590095)
>> My initial response was based on the assumption that all prompts used that script prompt quoted in the parent.

You, and another 20 or so commenters here. We should really re-examine the guideline about asking people to RTFA.

No offense meant- good on you for correcting your error.