That said, there are other moat factors like, a US company needing to use a US AI provider, sticky customers due to corporate onboarding friction, and others. Not nothing, but not as large a moat as some imagined.
There were somewhat good reasons to think it needed more than just this data-driven ML approach.
Then i think tool use became a priority or at lest a sibling priority to more data/more params. Along with multiple specialized models communicating with each other which is sort of a special case of tool use. That pretty much brings us to today.
Having a dude doing translations has always been useful even if they didn't understand the subject matter. Happened all the time pre LLMs
Here is a project that guides you through it if you want to prove to yourself that it works https://github.com/arcee-ai/DistillKit
GPU kernel optimization is just the kind of well-bounded problem with clear success criteria that AI loves.
It's about evidence this is an active force in competition in LLMs.
It's also how providers build their smaller models out of their larger ones; they publicly talk about the process.
Make sure to stay updated!
If we wanted to compare model responses, we would give all models system prompts with model names, thereby fixing the Kimi misattribution.
The reason Kimi often states its name as Claude is likely because we can actually run it without the mandatory system prompt, smoothing over awkward competitor mentions.
[1] https://www.anthropic.com/news/detecting-and-preventing-dist...
I’m really not sure how you’re getting “exact” here.
They are crying about theft after committing the largest theft in human history.
I believe it can be used in the RL / fine tuning sense, in which case 150,000 requests, assuming every one was detected, could move the needle in quality.
I agree it couldn’t replace all of pre and post training , but I don’t think that’s the claim. You do typical training, then distill really difficult cases.
He even raised money on that premise.
He is a pathological liar, so is Dario. Don’t rely on the benevolence or truthfulness of these people.
They will say whatever is beneficial to say in the moment.
Referring to a baseless prediction by Sam Altman that AI will become like electricity without any push-back? Who really thinks Sam is working toward that future?
He already worked to undo every early promise made (non-profit, open source models, strong governing board, strong ethics/alignment/security focus). He's flip-flopped on other things like first characterising Trump "an unprecedented threat to America", then contributing 1M USD to Trump's inaugural fund far exceeding his earlier political contributions. Lately OpenAI, under his supervision, has also been working with Anthropic to lobby regulators in Washington for restrictions on open weights models - why so if not to undermine a free market in favour of an oligopoly?
Beyond that, you have the simple fact that most of his personal wealth and very probably the fate of OpenAI hinges on AI inference NOT becoming an interchangeable commodity.
I mean.. Honestly. The naivete is downright astounding.
I live in Germany where people won't stop whining about electricity prices, and I pay 75€/mo.
China or SpaceX seem like the 2 likely candidates in 5 years, but who knows.
If (a) demand for AI continues to increase, and (b) SpaceX can get to ~$100/kg to orbit, then they will have a ridiculously deep moat. Probably more like 10 years, though.
But as you said, who knows.
You can put AI chips in datacenters in the desert for far less than $100/kg. With lots of solar power available, the option to easily access your hardware and far less radiation issues.
The datacenter in space story really only exists to make it possible for Musk to sell X to SpaceX and make more money from the IPO. That's all. There is no engineering reason.
Musk's bet is dysfunctional politics will make it impossible to build enough data centres and the energy needed to power them. There are many, many reasons that might be wrong. However, if the economics are even close to viable, they could start throwing up data centres quicker than anyone can build them terrestrially (at least in the democratic west).
GMAFB.
It's stock pump bullshit from a guy who has figured out how to extract the maximum from stock markets.
But yeah, I'm sure you people with Elon Derangement Syndrome actually have it all figured out /s
well, no, they announced the concept of a space-optimized Vera Rubin designed with SpaceX:
> NVIDIA and SpaceXAI are working to adapt that foundation to the requirements of orbital computing while preserving a common NVIDIA architecture and software ecosystem.
The press release is really announcing that SpaceX's terrestrial data centres are going to use Vera Rubin.
https://nvidianews.nvidia.com/news/spacexai-adopts-nvidia-ve...
if you can't put them outside of Amarillo Texas without people throwing a fit then you can't put them anywhere. I mean freaking Pantex is there ffs!
Cooling is probably the easiest problem to solve, easier than power. And in both cases, the problem is solved by mass to orbit. All you need for cooling is a big f-ing radiator. Solar panels are chips, and not trivial to manufacture. But a radiator is just a hunk of metal with some pipes.
That's why the cost of mass to orbit is the most important thing. You can solve almost any space problem by just throwing more mass at it.
Solar panels are 4x more efficient in space. 2x because of lack of atmosphere and another 2x because there are 24 hours of sunlight in SSO.
Is it really that hard to just say, "I don't know if it will work or not"?
Ridiculous. Have you ever seen a container ship? One engine on a container ship puts out 80 megawatts, or around 80 orbital data centers' worth of power. So if there's not enough room for solar, which would surprise me greatly, then running a generator is not going to be a problem.
As for the lack of 24-hour operation, a ~50% duty cycle is a perfectly reasonable tradeoff for not having to launch something into LEO. You can easily deploy several times as many installations for the same money if they don't have to work in space. And they will be maintainable, to boot.
And I fully sympathize with the need to remove these things from local and even national sovereignty, having seen as an increasingly-horrified American just how destructive populist influence can be.
But putting millions of servers in orbit? That just doesn't seem like the way forward, for so many reasons.
As for weather, OK, fine, whatever. Deploy 6x as many.
But anyone who thinks they can predict those prices in ten years is wildly overconfident.
> then they will never work and SpaceX will fail
Space Datacenters eventually get quietly discontinued after finally killing the zombie brand at which point the only reaction is "huh, remember that?"
https://techcrunch.com/2026/08/21/teslas-solar-roof-is-dead-...
ETA: If you had invested $1,000 in 2016 when Tesla Roof was announced, you'd have $25,000 today. 25x return in 10 years.
If it doesn't work out, I think China's exponential terrestrial energy deployment will eventually give them the lead, IF they can get enough chips. Another big if.
The economy he and his ilk want to build is infinitely worse.
If investors start fleeing from senseless businesses in the AI sector, that does not mean that sensible businesses will be spared. These things follow herd mentality, and the primary drivers of the herd are greed and fear, not fundamentals or business logic.
A major correction would be a bummer but we were never entitled to these abnormal gains in the first place.
Personally, I came to this conclusion early this year. To acquire the data that AI Companies are using to train their models is low cost and once they have it, they can refine and store it. Creating the LLM takes a bit of money but it is not a serious blocker. Clearly, the Chinese companies can make AI so they will drive down costs. There is a need for good AI (Not just Great AI) and it is not cost prohibitive to make good AI (The same with specialized AI).
My prediction is that AI will spilt into two categories, Great AI (High Cost) and Good Enough AI (Low Cost). Which for the long run of AI and companies that use AI, this is good.
Besides, identity verification that actually works at scale is a much harder problem than identity verification which is good enough to satisfy your compliance people and regulators. Especially if the fraudsters have a major world government standing behind them, and if their aim is to be identified as a real customer, not one customer in particular.
And with the sheer volume of data created from that, coupled with benign-seeming prompts like "plan out your reasoning in a document before implementing" that could never be patched without breaking existing customer workflows... there's more than enough for someone to distill on. Even if that only gets them to not-quite-frontier, if you're pushing the frontier every few months, they're only ever a few months behind you.
The primary resource you need to train LLMs is money and China has plenty of that.
Distillation was big news a year or even 6 months ago, but as far as we can tell it's not really a moat anymore. Now that multiple players have trillion+ parameter models and the capacity to post-train them, there's no putting the genie back in the lamp.
Same problem: if humans can see the output, it can be copied.
If I run out of tokens on ChatGPT of course I will try Claude. I never ran out of Google searches so no reason to try Bing
At some point Google gets suspicious of your persistent searches and makes you solve captchas and puts cooldowns on your searches.
More like the other way around - Claude burns tokens faster than any other LLM.
ChatGPT is AI for the average non-techie the world over, but the average non-techie isn't eager to pay for it. The more progress that's made, the less incentive to pay - most people are happy with the total garbage spewed by google AI overview. They'd be happy with google's 30b MoE gemma, whose performance will likely be squeezed down to something that can run on a phone in 2-3 years. Why would they pay $20 a month?
It's why OpenAI is pushing a variety of things such as ads and offer a more polished ui/ux than the competition, I think. The models are already good enough for people who just want to know how much sugar to add to their cake or when's the next basketball match their team plays - it's OpenAI's game to lose those people, by annoying UX and whatnot. If they can make a few bucks off of every one of their non-paying users it'll stretch their runway immensely. Those users will never go to Antrophic or some cheap Chinese model, but they might defect to Google because a popup on Android / in Chrome told them to.
I had a discovery call last week with someone who did not realize he could use ChatGPT for work. It was a revelation that he could drag a PDF into ChatGPT and it could summarize it for him.
FWIW, guy in his late-30s in a pretty senior sales role.
Wouldn't that just be price fixing? If they arrive at their prices independently and they all happen to be similar, fine. But if they're all "smart" and coordinate so none of them undercuts the other, that's probably illegal.
The frontier models are a replicator that can give you another replicator which specifically produces tea, earl grey, hot, when you push the single button, and does nothing else.
The quality of the harness UX, and random fun crap like Sora, it's a shame that OpenAI killed that so soon, and also Group Chats in ChatGPT.. they risk running a Googlelike reputation at this rate
Maybe ultimately whomever can be the "Apple of AI" will win
And I'm saying this as somebody that's made millions selling AI software in the last few years...
The Chinese models are pretrained on large clusters just like OpenAI ones are. Yes, they use outputs of the frontier models to further improve the final model, but even without those outputs they'd still have very strong models.
It's not like in a world without distillation things would be much different as you claim.
Tricks like distillation save compute in the RL leg of the process - where a lot of the frontier labs puts their own training run compute.
Transformers are like just a step or two removed from being fancy convolutional neural networks. I guess I'm just surprised that it didn't turn out to require more 'special sauce' with extremely elaborate internal architectures, and less of a big-data approach.
Because the data is so central in building these LLMs, rather than some special insights or ideas in the model architecture, or very special hardware requirements, the field is much more open than I would have guessed some years ago. And it's the fact that the data is so central that makes distillation possible in the first place.
Sort of. It means the country on the verge of monopolizing all aspects of hardware production (China) doesn't need to rely on outsiders for the software. So while that weakens one monopoly it strengthens another.
At a regular store not some weird nerd laptop for normies.
Can you buy a phone/laptop with no parts from USA/Japan/Netherlands/Taiwan/South Korea?
The times of China needing Nvidia chips is quickly coming to an end.
I guarantee that China will scale faster, build faster, and ultimately produce far more chips than the rest of the world combined in 5 years.
Our export controls sank the west. It would have been better to allow them to use Nvidia chips. Now they will have chip fabs that aren’t quiet as good but way more of them. Their investment into sustainable will make the power so cheap that the less efficient chips will not be a relevant issue
China has mastered lower parts of the value chain, but the most advanced and profitable ones (chips, turbines, frontier models) are solidly dominated by the US and its allies.
That might change, but when? The demographic and economic advantages that helped China rise quickly are exhausted. The Chinese economy is struggling to unwind the property bubble and manage a huge amount of surplus production that can no longer be sold to the US. The country is shrinking in population. Urbanization, which was previously a sure fire way to achieve growth, is now at ~70%, compared to the US' 80%.
It seems unlikely that China will be able to fully supplant the rest of the world unless the government relinquishes its tight control of the economy. There's just not enough centrally-plannable growth left.
The US will be (is) forced to work with everyone not named China or Russia, to ensure it can compete with China's concentrated industrial might. That means making iPhones in India, or Mexico, or Vietnam. That means Micron building more RAM production domestically. It should mean helping ASML stay out in front of China's incoming silicon wave. And so on.
It'll be China + allies vs the US + allies. It's going to be intense and very expensive. A huge amount of global manufacturing has to be built up outside of China.
And China does indeed have enormous problems brewing, including a staggering 5x increase in homelessness in just roughly seven years (to go with its collapsed housing market):
https://www.upi.com/Top_News/World-News/2026/01/14/beijing-b...
Over the next 20-30 years robotics + AI will brutalize countries with very large populations when it comes to employment, especially those that depend on volume manufacturing heavily to absorb labor. The US for its part should be focused on rapidly shrinking its population, pushing up-skill, restricting immigration based on points (ala Canada et al.), and focusing entirely on quality of life / standard of living per capita as the population declines (let it happen rather than fighting the big trend, boost GDP per capita via robotics + AI, higher productivity).
I'd want to be a small, decently well educated, high functioning nation, with powerful military friends, and far from China. Estonia perhaps (Russia poses no serious terminal threat to them so long as NATO exists or a European alternative). Poland has some cultural mojo right now as well, they could be interesting over the next few decades.
Interesting to hear a new take. Not what you commonly hear. Most focus on the damage caused by a shrinking and aging population. What's your opinion on that situation?
I would think the trick will be the figure out how robotics + AI will enable a new level of elder care, but it's hard to imagine robots being delicate and nuanced enough to do that kind of work.
Its not that wild of an idea. It's not like e.g. chipmaking where even just knowing how things are done doesnt mean you can copy it.
They disabled the temperature and seed parameters. There's still logprobs, so they aren't as closed up as Anthropic yet.
I might write about an article of the history of LLM APIs, I used to think the ChatGPT was going to be a de facto standard like intel's 80866 mutated into x86, but it seems to be a bit more nuanced and diverse than that, vibecoding introduced so much complexity because the vibecoding product itself became vibecoded so the enshittification was accelerated, many such cases.
Your seeing the AI labs respond by never publishing chain of thought now and in the future, I see them not even publishing their top models as a general purpose API and instead using it to drive their own AI apps, which will obscure even more model output. Anthropic Mythos was internal only for many months for example.
They're not, right? If it's really easy why there are no counterparts of DeepSeek from the Europe or Japan?
GPT 5.6 Luna at $0.20 per 1M is pretty good for 80% of enteprise applications.
Probably not through what we call distillation now. There’s some magical technique hiding there, go find it!
Why shouldn't it work though? It's just models teaching other models same way humans are.
10 or 15 years ago if one had asked me to envision a future where a private company invents artificial intelligence, I'd have thought for sure they'd have a massive moat, be very difficult to catch, and it would create an almost instant monopoly.
Rather, it seems that selling intelligence might end up as a race to the bottom.
Who woulda thought that just having access to enough textual inputs and outputs and a vaugely similar transformer architecture would be enough to copy-cat rather useful intelligence.