m_ke

1. abundance of digital goods

2. automation of computer work

3. commoditization of "intelligence "

4. collapse of frontier labs

humans will be fine, we'll have rapid progress on anything a human could do on a computer but it will take a really long time to trickle into the real economy

You have to understand that Gates had other distractions to deal with

just like there were mistrals, coheres, llamas, etc, there will be new deepseeks and moonshots if those ever flame out (worst case, given out at cost by google, meta, alibaba or etc)

OpenAI and Anthropic are already in a ~200bil hole from previous model iterations and are committing to trillions of additional spending

OpenAI spent more TBPN than kimi spent on training K3

So how exactly is Anthropic and OpenAI ever going to pay back the trillions that they plan on spending?

With spec decode should easily get to >100tps

on my dual 3090s qwen 3.5 27b was running at around 110tps using the config from https://github.com/noonghunna/club-3090

make that 200tps on a single 5090, 4x faster than opus https://x.com/radixark/status/2088285681131110446

devs about to get handed a two 5090 box each and told to max that out

when you're waiting for network or LLM inference the raw performance doesn't matter at all

1. it's built for async 2. runs everywhere 3. interpreted, making it fast to iterate on 4. decent performance 5. most popular language, llms are decent at writing it

People are not ready for what will happen once they start producing their own chips and no longer have to pay the 80% markups on nvidia.

And it will get even worse in 2-3 years when it's time to compete in robotics.

All of these harnesses should support pinning config files and tools to specific models or families of models.

It's really tiring to have to tweak everything with each model release and then watch those changes mess up cheaper models in the process.

GLM 5.2 feels better than Opus and K3 is as good as Fable.

Now I can't wait for someone to distill K3 into a Qwen 3.6 27b or Poolside S 2.1 sized models for a proper fast local Composer 2.5 replacement.

Yeah some real main character energy from Dario as usual.

I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.

the argument is that we should all fold and let sam altman burn trillions of dollars on naive scaling and pay monopoly prices for their closed APIs until the models are good enough to be closed off for "safety" reasons so that they can take an even larger cut by competing directly with us

Oh this definitely happens all the time. I was an early employee at Clarifai, which won imagenet a year after alexnet and we were able to stay on the frontier for about 2 years before a bunch of open source models were matching our results. It was always some random PhD research project spinout or some random kid in Boston named Alec Radford.

We had a bunch of things that we never published that ended up being major research findings years later at top conferences.

only if you only get your news from main stream business press and Big Lab propaganda channels

There's no chance K3 is a distill of Fable, it came out way too soon after the limited fable release to be feasbile.

If you look at all of the top ML conferences, chinese labs contribute way more to advances in ML than "Open"AI and Anthropic: https://www.reddit.com/r/TheMachineGod/comments/1pi4q7f/pape...

This K3 release just helped every other lab on the planet stay in the race by making it possible for them to build on top of it, placing them at the frontier starting line instead of having to spend billions of their own dollars and risking it all to attempt to catch up.

The open source contributions I linked to above will move the whole field forward and reduce the costs of training and inference for everyone.

Open science compounds on it self, every new advancement pushes the field forwards and opens up new grounds for future improvements.

It is impossible for a single closed lab to consistently stay ahead of the rest of the field, especially in a huge growing research area like machine learning. The only only advantage the big labs have is money, but the naive scaling game is not sustainable long term when you have to pay 10-100x more then the fast followers and we start getting more and more open models or use case specific models that can handle 90% of high volume use cases.

Research is a high variance, low expected value activity, meaning that the few large concentrated labs have to be conservative with their bets and double down on proven things when scaling up. The rest of the field is like a diversified portfolio, with thousands of players making smaller riskier bets that only require a few of them to succeed (like K3 did here, and DeepSeek a year ago)

EDIT: also if you look at most of the work from OpenAI, it's mostly taking existing promising open research work and scaling it up. (except for things like CLIP and etc from Alec Radford)

no, the goal was to spark a conversation about the value of *open AI* and it looks like it worked

Also open sourced a bunch of infra to go with it.

Anyone who claims open source and open weights models are "decel" needs to get their head checked

https://github.com/MoonshotAI/MoonEP

https://github.com/kvcache-ai/AgentEnv

https://github.com/MoonshotAI/FlashKDA

On Policy Self Distillation and Active Learning. Anything that increases sample efficiency by providing a richer more dense feedback signal and is more efficient at exploration / sampling.

I wonder what they'll do when there are open european and american models that lap them. Will we see "Open"AI claim open ai is a threat to humanity?

Anthropic should think hard about all their fear mongering. It will only end up backfiring on them and everyone else involved.

They definitely used closed private saas products to train their own models, to prove that just drop random small screenshots of any popular product behind a login screen and see how well it's able to identify all of them. ex: https://x.com/michalwols/status/2079968211865330165

or other similar "AI" startups https://x.com/envconfig/status/2079613455296827402

I assumed google would lean into the efficiency stuff more and try to eat the easy 80% of workloads, winning market share on volume instead of frontier if they were not able to produce frontier level models.

They're very well equipped to be the volume discount store of inference.

k3 costs will go down at least 3x within a week of the weights dropping.

we'll get new quants, dspark speculators, distills and optimized kernels

as long as there are near frontier models available there will be inference providers selling them at or below cost of inference in attempt to get market share.

Sure, but anthropic is charging businesses based on usage now and tried hard to pull Fable from the consumer subscriptions before Sol and K3 dropped.

Even now on the $200 plan I use up my Fable credits in a single day and had to start using codex and openrouter for more usage because Fable burns $100s an hour when billed on usage.

Anthropic will get squeezed by open models for 80% of the use cases that don't require frontier capabilities and by vertical specific labs for the high value tasks that would (bio, finance, math, etc.), where smaller use case specific models will beat them on cost and speed while matching or exceeding the performance of their largest general models.

Even their hail mary of being first to "AGI" will never happen because all it takes is China blockading Taiwan or Nvidia cutting them off to stop them from eating up a large chunk of the economy.

There is no scenario where the rest of the world will sit on their toes and let OpenAI or Anthropic monopolize "AI". Too many countries, large well capitalized players and partners / suppliers who could never let that happen.

Kimi K3 allows all existing players to restart at the frontier and keep competing with OpenAI/Ant. It also gives employees at these labs a better more lucrative path of starting new labs with fresh books and clean cap tables, building on top of K3 without needing to spend all the capex on pretraining their own models. Plenty of them already vested their stock and would have 0 problems raising 100s of millions of dollars for new labs, making them paper billionaires over night.

most of the gains from the past year and a half have not been from web data, but from synthetic data and agent rollouts with RL.

yeah I tried rules, hooks and forbidding things like pip / python3 but it just led to the model failing to do what it wants and a bunch of token churn trying to get around my more rigid constraints.

main problem is that the harness files get loaded into context early in the session and slowly wash away as new information comes in.

It's not an issue of usual vs unusual, I'm saying the models are way better at writing and maintaining django or react code bases than your own hand rolled architecture that you define in some docs that it has to learn and keep in context. All of the models do an amazing job making local edits and working in small greenfield projects but once you get to full production systems with close to a million lines things start to rot. The code still works and the models are able to make progress but all of a sudden you have 3-4 different versions of your concepts sprinkled in random corners of your code base.

My second point is that the models are way better at things like Rust or Lua than Python or JS, because the average person producing code in those languages has way more programming experience, so the code quality of training data online in those languages is higher than the programming 101 medium blogspam type content you see in more popular intro languages.

I tried really hard to do this, but it turns out the models don't care about your ideas and want to do what's popular in their training data, so they will happily ignore anything you try to force down their throats, especially as context length grows or if you hit compaction.

So to make best use of the models steer them down familiar paths, mention common pattern and frameworks, use popular packages and languages that have the high median quality online.

I started my project with a few simple interface definitions and a short design / architecture doc that I include in the AGENTS.md file, but no matter how hard I try all of the models just end up ignoring it and sprinkled new seeds of variants of the same stuff all over my code base, that with each new session grow new branches.

With agentic coding the costs of tokens compound with each message / tool call and etc. Having to load in and update large files makes things slower and way more expensive.

Databricks actually just posted some of their own benchmarks on how harness alone impacts costs https://www.databricks.com/blog/benchmarking-coding-agents-d...

simple things like passing more file context, model having to explore the code base at start of each session, writing comments or markdown docs ends up increasing, running into test / build issues can 3-10x your costs.

PS: my code is still mostly TS and rust but I'm considering moving some of my annotations into .d.ts files and having them generated from runtime types (ala MonkeyType).

After running out of Fable credits in a day on my max plan I started looking around for ways to trim down my token usage and came to the realization that all of the type spaghetti that opus wrote is probably eating up like 50-70% of my tokens.

A clean django project is probably 3-4x less code than the equivalent TS based service.

It made me consider dropping strict mode and defaulting to js for most simple things.