10% chance to be market leader doesn’t sound as good as 60% second place to them for example
It's his call for sure, but rather funny as an argument.
It is because even if it would pan out, it would mean temporarily losing AI leadership, reducing stock price, losing the ability to hire talent, and by virtue of that losing the opportunity permanently.
He was against LLMs, reinforcement learning, etc. Claiming that those are a dead-end. He was wrong. His own "JEPA" ideas have basically gone nowhere. Meta was right in moving on without him.
Zuckerberg wanted to placate his shareholders with a shiny new thing now.
would've stopped Meta from delivering a polished product like Muse sooner.
Muse is not polished, unless usability and programming standards have fallen so far in the LLM vibe coding era that a pre-alpha state is now considered polished.
Muse is imperfect, as is all software, but calling it pre-alpha is a bit much.
But even in computing, when there is a promising way forward that the basic reaearch has been done on, its hard to justify side quests.
...And because they have monopoly money. Basically, research labs are usually vanity spending for corporations with too much money. Xerox parc was funded by a monopoly. Bell labs was funded by a monopoly. Google funds its moonshots with its monopoly.
How does that work when the research team moved elsewhere? Re-hire talent? That takes time - the 1 thing you don't have in a fast-moving field like AI. Post sums it up nicely:
"Do the math—how much is it worth to halve training and inference cost given that the industry has investments approaching a trillion dollars?"
How much is these researchers' salaries, compared to what MSFT spends on compute? Chances are it's peanuts in comparison.
Companies will try to ruthlessly rid themselves of labor as long as those members of labor have agency and rights.
If someone disagrees they can convince me by explaining why a company wouldn’t choose to spend X on a machine they wholly own vs X on an employee they don’t.
There’s some room for argument that employees can do more than machines can but in the current AI zeitgeist arguing that we have machines with equivalent to human intelligence I don’t see how you wouldn’t see this coming if you believed the zeitgeist.
In a way this suggests that MS doesn't aim to be a leader in AI <the core tech>. Only in its applications + related services (read: profit to be had there).
Nothing wrong with that btw. But understandable that fundamental researchers then pack up & leave.
Ah, well, good thing we've almost entirely defunded academic science.
Seems weird when it's become apparent that LLMs aren't really special sauce.
The author states "I consider the scaling-only focus to be both misdirected and not fun.". I get the point but also totally get the focus on scale out for now.
For a corporation it is to make money. For a researcher it is to explore and illuminate. That exploration and illumination often leads to a landfall of new wealth generation is a happy accident that ensures that the monies are going to bet some of their capital in them, at least the wise ones.
Which is superior, the big f*king monstrosity or the elegant on-demand, small batches, reduce waste?
Meta will go in the shitter and Deepseek will make the small batches.
It does (Lean at least) incorporate controlled experimentation, but tightly controlled with a both a laser focus on what the goal is and tight resource constraints. And Lean (at least several writers on Lean) favors taking multiple paths with a a quick and low risk and a longer-payoff, higher-risk, higher reward path for some efforts—one source used the US Navy’s two-path approach to SLBMs with Polaris and Poseidon as a model for this—but that's different than “maybe there is something down this path” basic research, which is outside of the concern a production model like TPS/Lean and often completely outside the model of a product-oriented firm; basic research is something firms consume and incorporate more than generate, because until the basics have been proven out, the rationally perceived chance of any particular avenue paying off is near zero for any firm.
Meta will go in the shitter and Deepseek will make the small batches.
RCA developed the first good color TV cameras. They were monsters. Originally, electronic TV (as opposed to mechanical scanning) used Zworykin's iconoscope. This needed huge amounts of light. It didn't have a photo-multiplier stage to boost sensitivity, but it did accumulate charge from light over the whole frame time. There was an alternative approach, Farnsworth's image dissector, which also needed huge amounts of light. It had a photo-multiplier stage but only the light from the dot currently being scanned by the scanning beam mattered.
So RCA threw money at the problem. They developed the image orthicon camera tube. It combined the advantages of both approaches, integration over a full frame and a photo-multiplier stage. It was insanely expensive, costing thousands of dollars in the 1950s. It needed about twenty different voltages and multiple independent power supplies. It worked reasonably well.
Then came color. That took three image orthicons (RGB) inside a box the size of a small refrigerator, along with optics to split the incoming image into three colors. Plus a rack of analog electronics per camera, connected through a large cable. Plus an alignment problem from hell, with a huge number of adjustment knobs.[1]
Scaling worked! From the 1950s to the 1970s, color TV broadcasting was powered by those monsters.
Now, independent of this, other people were working on other approaches. Vidicons, silicon detectors inside camera tubes, and finally fully solid state image sensors. Eventually, in 1983, after thirty years of image orthicon monsters, RCA produced a usable solid state camera. Today, a cell phone camera costs about US$10 in bulk, and it's far better than the monsters.
It took thirty years of low-priority development to get past the giant image orthicon cameras.
Electronic fluorescent lamp ballasts had a different problem. That was a suppressed invention. The magnetic ballast industry was dominated by Magnetek. Someone invented a solid state replacement, which was smaller and cheaper, and licensed their patent to Magnetek for royalties. Magnetek just sat on it, not wanting to gut their core market. The inventor hired Townsend, Townsend and Crew in Palo Alto and sued. (Read this in the law firm's brag book in their office.) Eventually, the inventor got a hundred million or so, solid state ballasts went into production, and Magnetek seems to be gone. (The company called Magnetek today, which makes crane controls, is an offshoot of a Litton unit, and seems to be unrelated.)
There are also parallels from the steam locomotive industry, the Baldwin Locomotive Works being the best known. Baldwin actually built the first Diesel locomotive. They made a few half-hearted tries at Diesels over several decades, then gave up and went back to steam. Baldwin ceased all locomotive production in 1956. The corporation lingered on for a while, not doing much.
[1] https://www.tvcameramuseum.org/rca/tk40a/pix/rcatk40aart.pdf
Suppressed inventions are very rare, but that's one of them.
One of the few others is Airadar, from 1973.[1] Not "AI", "air radar". This was a phased-array radar for small aircraft. Way ahead of its time. So far ahead that the USAF was upset about it being available commercial before it was in use by the military.
[1] https://books.google.com/books?id=NWzlTqj0gQ4C&pg=PA64&lr=&r...
According to official data from the U.S. Patent and Trademark Office (USPTO), there were 6,653 active secrecy orders in effect at the end of fiscal year 2026. Roughly 100 to 350 new secrecy orders are added every year. Many remain locked away for decades or more.
If the U.S. government puts a secrecy order on a patent application against the inventor's will, they are legally forbidden from talking about it. The inventor cannot talk about how it works, publish research on it, sell it, or market it. They cannot even discuss the exact classified details with a regular lawyer unless that attorney has the proper security clearance.
The rest is history.
That failed and SpaceX was then forced to develop its own brand new engines.
Where can I read more about this? I have seen people make this claim for a while, and when I looked deep, it used to turn out to be wrong.
Lex fridman podcast 2022
ChatGPT answers this just fine.
He also said they will be “ Writing aids, not much more" and they “Can't plan.”
No one said he knows nothing about building AI. He was just clearly wrong about LLM capabilities.
As far as I know, those things are still true. Unless there is text in the training material that the text in the prompt can map to, or there is text in training material that describe how to map text in the prompt to text in the training material, LLMs cannot process the prompt.
So may be he is wrong about what could be in the training material. But I don't think he is wrong in the fundamental..
This is sort of what I mention in my original comment. These claims, when pushed, turns out not to hold water...
AGI will require architecture changes not simply more data.
At no point do I ever confuse a human being with a Claude session.
Even if I'm somehow fooled or confused for a minute, the context window / hallucinations / Claudeish will immediately give it away.
Sure, we passed the turing test for chatbots but we are very far away from AGI.
We did?
> This is also taking his comments about transformers scaling out of context
> AGI will require architecture changes not simply more data.
[0] He speaks about Bitter Lesson directly at 9:57 but the earlier part gives better context: https://www.youtube.com/watch?v=21EYKqUsPfg
Yann LeCun has the syndrome where someone who sees the value of B (non-text modalities, i.e. physical reality) cannot stop themselves from constantly critiquing A (LLMs), even though clearly A has its own advantages.
I think he is a smart guy. But he constantly deploys the crackpot's PR playbook.
More specifically:
* he said that autoregressive transformer is fundamentally flawed: if it makes one error, all further generated tokens make no sense; that's, obviously, bullshit: there are many ways to implement backtracking, and there are many papers demonstrating backtracking at a time he did that lecture. And, of course, it just doesn't matter - reasoning model can just write "Oh, that's wrong. Let's try a different thing". - it's just not derailed by one mistake.
* he said that GPT-1000 can't possible have understanding of physical world, when GPT-4 already existing at that time could solve a problem he described.
So it's not like he was bad at predicting things, he's intellectually dishonest.
The idea that someone should be fired just because they were wrong seems to be something that would cause issues if one wanted to think long term.
This approach will make people afraid of being wrong. It will lead them to avoid challenging dogmatic assumptions in large organizations and inevitably lead to group think.
People really shouldn’t be punished for saying wrong things. Long term it’s better for a company if they aren’t punished for that. Helps to let people freely criticize dogmas or propose new ones.
You're being uncharitable. He would have been right if Chain of Thought hadn't been pulled out of a hat, and no one saw it coming.
At this moment no one can know if there will be a future step-change on LLMs similar to reasoning, and conversely, no one can be absolutely sure there won't be - so it shouldn't be a firable offense because it's not obvious; even though it looks like it in hindsight.
Even a nobody like me was playing with a sort of CoT with llama finetunes before o1 came out.
Only this year I'd say we picked all the low hanging fruit that has been obvious for a while that was needed.
For verifiable tasks like coding and math, you can keep going, doing more and more RL. You have the AI generate hard problems, then try to solve them and reward correct solutions. We basically have RSI for verifiable tasks, we just need to keep the wheel turning.
Meta AI is actually frontier lab now with LLMs. Muse Wonderfull is usable with multi agent orchestration (where other labs struggle).
I do not think that would happen with pure research and without single focus!
Almost all statements Yann LeCun made back then (LLMs won't be able to do this or that) turned out to be wrong. They were right to fire him.