News for hackers. Stuff that matters.
Re:Claude 4 by waleedlatif1 on Thursday May 22, 2025 @01:12PM (#44064194)
I really hope sonnet 4 is not obsessed with tool calls the way 3-7 is. 3-5 was sort of this magical experience where, for the first time, I felt the sense that models were going to master programming. It’s kind of been downhill from there.
Re:Claude 4 by lherron on Thursday May 22, 2025 @02:43PM (#44065309)
Overly aggressive “let me do one more thing while I’m here” in 3.7 really turned me off as well. Would love a return to 3.5’s adherence.
Re:Claude 4 by nomel on Thursday May 22, 2025 @03:25PM (#44065784)
I think there was definitely a compromise with 3.7. When I turn off thinking, it seems to perform very poorly compared to 3.5.
Re:Claude 4 by mkrd on Thursday May 22, 2025 @04:53PM (#44066847)
Yes, this is pretty annoying. You give it a file and want it to make a small focused change, but instead it almost touches every line of code, even the unrelated ones
Re:Claude 4 by artdigital on Friday May 23, 2025 @04:53AM (#44071151)
Oh jeez yes. I completely forgot that this was a thing. It’s tendency to do completely different things “while at it” was ridiculous
Re:Claude 4 by Jabrov on Friday May 23, 2025 @05:55AM (#44071423)
Can be solved with more specific prompting
Re:Claude 4 by deciduously on Thursday May 22, 2025 @01:24PM (#44064330)
This feels like more of a system prompt issue than a model issue?
Re:Claude 4 by ketzo on Thursday May 22, 2025 @03:05PM (#44065548)
Anecdotal of course, but I feel a very distinct difference between 3.5 and 3.7 when swapping between them in Cursor’s Agent mode (so the system prompt stays consistent).
Re:Claude 4 by waleedlatif1 on Thursday May 22, 2025 @01:37PM (#44064498)
imo, model regression might actually stem from more aggressive use of toolformer-style prompting, or even RLHF tuning optimizing for obedience over initiative. i bet if you ran comparable tasks across 3-5, 3-7, and 4-0 with consistent prompts and tool access disabled, the underlying model capabilities might be closer than it seems.