Re:The End of AI Frontiers
by killerstorm on Saturday November 07, 2026 @09:21PM
(#50409922)
It's not so much they _turned_ wrong, they were obviously wrong at the time he made statements, or shortly after. He is just extremely biased against transformers, and it's easy to see why - transformers are already invented, and as a scientist he wants to invent something new.
More specifically:
* he said that autoregressive transformer is fundamentally flawed: if it makes one error, all further generated tokens make no sense; that's, obviously, bullshit: there are many ways to implement backtracking, and there are many papers demonstrating backtracking at a time he did that lecture. And, of course, it just doesn't matter - reasoning model can just write "Oh, that's wrong. Let's try a different thing". - it's just not derailed by one mistake.
* he said that GPT-1000 can't possible have understanding of physical world, when GPT-4 already existing at that time could solve a problem he described.
So it's not like he was bad at predicting things, he's intellectually dishonest.
More specifically:
* he said that autoregressive transformer is fundamentally flawed: if it makes one error, all further generated tokens make no sense; that's, obviously, bullshit: there are many ways to implement backtracking, and there are many papers demonstrating backtracking at a time he did that lecture. And, of course, it just doesn't matter - reasoning model can just write "Oh, that's wrong. Let's try a different thing". - it's just not derailed by one mistake.
* he said that GPT-1000 can't possible have understanding of physical world, when GPT-4 already existing at that time could solve a problem he described.
So it's not like he was bad at predicting things, he's intellectually dishonest.