News for hackers. Stuff that matters.
Re:GPT-5.6 by bevekspldnw on Thursday July 09, 2026 @02:43PM (#48850580)
These aren’t raw base models they are the result of a ton of RLHF and various adjustments.

Bitter lesson wildly overstated in this context.

Re:GPT-5.6 by froh on Thursday July 09, 2026 @03:38PM (#48851339)
rlhf = reinforcement learning from human feedback

(had to look it up)

Re:GPT-5.6 by visarga on Friday July 10, 2026 @09:17AM (#48859525)
I think it's more RLVR (reinforcement learning from verified rewards). The RLHF is just to align models to human preferences, meaning to behave nice.
Re:GPT-5.6 by redanddead on Friday July 10, 2026 @10:07AM (#48860158)
What makes you say that
Re:GPT-5.6 by versteegen on Friday July 10, 2026 @10:52PM (#48868213)
More accurate to say RLHF aligns models to human preferences, most significantly to be helpful.
Re:GPT-5.6 by fastball on Thursday July 09, 2026 @04:00PM (#48851627)
More RLHF is in fact scaling.
Re:GPT-5.6 by bevekspldnw on Thursday July 09, 2026 @05:09PM (#48852411)
Yes, but not in the “dump another chunk of all written language in the bucket and stir”-sense which is what bitter lesson became synonymous with.

That may not be the intent of the original article, but over the past few years that’s what the phrase turned into.

Re:GPT-5.6 by avarun on Thursday July 09, 2026 @05:59PM (#48852932)
GPT-6 is supposed to be using a much larger base model that just finished pretraining so the "dump another chunk of all written language" approach is still going strong.
Re:GPT-5.6 by linkregister on Thursday July 09, 2026 @07:34PM (#48853855)
Modern pretraining also consists of expensive human-led specialized task creation and grading loops. Synthetic generation and distillation from previous models is another input for training. I wonder how much new text contributes beyond keeping knowledge up-to-date.
Re:GPT-5.6 by zamadatix on Thursday July 09, 2026 @10:42PM (#48855143)
The measure is to see if the results scale, not just the rumored attempts at building such a model, as o3 taught us.
Re:GPT-5.6 by Wowfunhappy on Friday July 10, 2026 @07:49AM (#48858656)
Where are they getting new data from at this point? Didn't they already read the entire internet?
Re:GPT-5.6 by redanddead on Friday July 10, 2026 @10:09AM (#48860192)
Custom sets and mining their own users, as every lab does
Re:GPT-5.6 by bevekspldnw on Friday July 10, 2026 @03:16PM (#48863992)
Custom what though? And user queries are not a knowledge repository it’s an ignorance repo.
Re:GPT-5.6 by krackers on Friday July 10, 2026 @04:53PM (#48865100)
A lot of data these days is synthetically generated. As an example, to make a model good at understanding assembly you simply need to round-trip code through a compiler and disassembler, then train against the source of truth and the assembly. You can generate arbitrary algebra expressions and have it solve it.

A lot of pretraining is also choosing the right type of data, you don't want to just have it ingest garbage (although I read that some amount of garbage actually helps the model be more robust). Pretraining crystallizes a lot of the inductive biases that post-training builds on, so by crafting the right data mixture you can make it easier for it to start off with a good foundation. There is also a lot of focus on mid-training these days, which I understand is basically either the name for the synthetic data stage, or the SFT phase before all the RL

Re:GPT-5.6 by bevekspldnw on Friday July 10, 2026 @03:14PM (#48863971)
Where did they get a second copy of all books, written/audio/visual media, and the web?

Couch cushions?

Re:GPT-5.6 by theptip on Thursday July 09, 2026 @10:53PM (#48855212)
The bitter lesson just means “compute scaling beats hand-tuned architectures in the long run”.

As GP said. More RLHF is in fact the bitter lesson.

Re:GPT-5.6 by cma on Thursday July 09, 2026 @04:11PM (#48851746)
The scaling with reasoning models is more and more with things like verifiable rewards (coding and math), in line with bitter lesson and also Sutton invented lots of modern RL.
Re:GPT-5.6 by theptip on Thursday July 09, 2026 @10:51PM (#48855202)
Nah, the last few generations have more RLVR in the data mix. Which is more CPU intensive and very much amenable to the bitter lesson as you can reduce the loss by doing more rollouts in your tool environment.
Re:GPT-5.6 by chaos_emergent on Thursday July 09, 2026 @11:11PM (#48855314)
RLHF is an increasingly small part of training though? From what I understand most of the capability gain is in RLVR
Re:GPT-5.6 by snthpy on Friday July 10, 2026 @01:36AM (#48856106)
Not just RLHF but also RLVR, and isn't that the litter lesson though?

My sense of the Sutton Dwarkesh interview was that he was calling out that he didn't mean just longer datasets, but rather learning through exploration and that's exactly RL.

Re:GPT-5.6 by andai on Friday July 10, 2026 @02:20AM (#48856391)
They just need more contact with reality. That's what RL is right? Contact with narrow subsets of reality.
Re:GPT-5.6 by 10xDev on Friday July 10, 2026 @09:17AM (#48859523)
Not sure where I implied they are "raw base models" and not sure what "various adjustments" means here or how "ton of RLHF" contradicts anything. If we look at research for open source models, "adjustments" usually come in the form of efficiency gains which directly contributes to the ability to scale or synthetic data pipelines to increase the dataset and increasing the context window.