News for hackers. Stuff that matters.
Re:Claude Sonnet Will Ship in Xcode by jama211 on Thursday August 28, 2025 @10:59PM (#45059650)
“Boycott” is a pretty strong term. I’m sensing a strong dislike of ai from you which is fine but if you dislike a feature most people like it shouldn’t be surprising to you that you’ll find yourself mostly catered to by more niche editors.
Re:Claude Sonnet Will Ship in Xcode by isodev on Thursday August 28, 2025 @11:17PM (#45059762)
I think it's a pretty good word, let's not forget how LLMs learned about code in the first place... by "stealing" all the snippets they can get their curl hands on.
Re:Claude Sonnet Will Ship in Xcode by astrange on Friday August 29, 2025 @12:37AM (#45060251)
And by reading the docs, and by autogenerating code samples and testing them against verifiers, and by paying a lot of people to write sample code for sample questions.
Re:Claude Sonnet Will Ship in Xcode by troupo on Friday August 29, 2025 @02:07AM (#45060745)
Yeah, none of that happened with LLMs
Re:Claude Sonnet Will Ship in Xcode by khafra on Friday August 29, 2025 @02:48AM (#45060994)
https://openai.com/index/prover-verifier-games-improve-legib... OpenAI has been doing verifier-guided training since last year. No SOTA model was trained without verified reward training for math and programming.
Re:Claude Sonnet Will Ship in Xcode by troupo on Friday August 29, 2025 @02:58AM (#45061047)
Your claim: "by reading the docs, and by autogenerating code samples and testing them against verifiers, and by paying a lot of people to write sample code for sample questions."

Your link: "Grade school math problems from a hardcoded dataset with hardcoded answers" [1]

It really is the same thing.

[1] https://openai.com/index/solving-math-word-problems/

--- start quote ---

GSM8K consists of 8.5K high quality grade school math word problems. Each problem takes between 2 and 8 steps to solve, and solutions primarily involve performing a sequence of elementary calculations using basic arithmetic operations (+ − × ÷) to reach the final answer.

--- end quote ---

Re:Claude Sonnet Will Ship in Xcode by khafra on Friday August 29, 2025 @03:42AM (#45061306)
My two claims:

1. OpenAI has been doing verifier-guided training since last year.

2. No SOTA model was trained without verified reward training for math and programming.

I supported the first claim with a document describing what OpenAI was doing last year; the extrapolation should have been straightforward, but it's easy for people who aren't tracking AI progress to underestimate the rate at which it occurs. So, here's some support for my second claim:

https://arxiv.org/abs/2507.06920 https://arxiv.org/abs/2506.11425 https://arxiv.org/abs/2502.06807

Re:Claude Sonnet Will Ship in Xcode by troupo on Friday August 29, 2025 @08:29AM (#45063209)
> the extrapolation should have been straightforward,

Indeed."By late next month you'll have over four dozen husbands" https://xkcd.com/605/

> So, here's some support for my second claim:

I don't think any of these links support the claim that "No SOTA model was trained without verified reward training for math and programming"

https://arxiv.org/abs/2507.06920: "We hope this work contributes to building a scalable foundation for reliable LLM code evaluation"

https://arxiv.org/abs/2506.11425: A custom agent with a custom environment and a custom training dataset on ~800 predetermined problems. Also "Our work is limited to Python"

https://arxiv.org/abs/2502.06807: The only one that somewhat obliquely refers to you claim

Re:Claude Sonnet Will Ship in Xcode by jama211 on Friday August 29, 2025 @02:56PM (#45068003)
Ah the classic “I don’t want to acknowledge how right that person is about their point, so instead I’ll ignore what they said and divert attention to another point entirely”.

You’re just angry and adding no value to this conversation because of it