News for hackers. Stuff that matters.
Re:Highlights from the Claude 4 system prompt by mns on Tuesday May 27, 2025 @03:38AM (#44104746)
I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and new "models" presented are them just editing and creating more and more complex system prompts. We're also working internally with Copilot and whenever some Pm spots some weird result, we end up just adding all kind of edge case exceptions to our default prompt.
Re:Highlights from the Claude 4 system prompt by Flemlo on Tuesday May 27, 2025 @05:18AM (#44105254)
But that's part of the human based reinforcement learning.

It now just happens on a way bigger level because now it's actually worth while to do.

That's one of the core beauties of the AI Ara

Re:Highlights from the Claude 4 system prompt by antupis on Tuesday May 27, 2025 @06:53AM (#44105677)
I think we already hit somekind of performance wall begin of this year. It feels that models are now balancing between rule following and agentic case and general stuff. eg Claude 4 sonet just feels better in Cursor and follows rules very well, and same time it gets equal or worse scores in benchmark against 3.7 Sonet.
Re:Highlights from the Claude 4 system prompt by merksittich on Tuesday May 27, 2025 @09:47AM (#44106974)
Speaking of performance wall: The Claude 4 results were added to the Aider LLM Leaderboard [0] yesterday. Opus 4 is clearly below Gemini 2.5 Pro at almost twice the price. Sonnet 4 fares worse than Sonnet 3.7, with the thinking version of Sonnet 4 being somewhat cheaper than its 3.7 counterpart.

[0] https://aider.chat/docs/leaderboards/

Re:Highlights from the Claude 4 system prompt by smusamashah on Tuesday May 27, 2025 @04:08PM (#44110256)
There might be 4.1 soon to make up for it's shortcomings.