Harnessing the Universal Geometry of Embeddings https://arxiv.org/abs/2505.12540
Yes, all three together would be even better. But it wouldn’t be if you had 100x more fanfiction, mostly synthetic, generated during RL to teach a model to be better at writing fan fiction. There are real limits to the amount of knowledge you can cram into fixed-size (downstream of hardware availability) weights. For a period scaling with data was basically “free” because we had the Internet and all the books/media that humans had already created; the data was accessible and limited (at least, the parts we think models should know about) enough and top-hardware big enough that we could basically compress the whole thing.
Post-training/RL are making this obsolete because they’re more about skill/capability acquisition rather than knowledge. They can generate much more data (most of it quotidian/useless, ie an agent made a typo in batch 382829) and clearly seem to cause a kind of mode collapse even in the most advanced frontier models.
We don’t need to make LLMs forget about SpongeBob SquarePants so they learn more about bash. But if I have a question about SpongeBob SquarePants, I don’t need to hear about load bearing seams prefaced with honest caveats after a model writes 400 lines of bash to look up SpongeBob’s family.
And there is probably a lot more SpongeBob knowledge we could put into models if we wanted to: interviews with the creative staff, a SpongeEnv/SpongeHarness modeling how the art/story team work together to create entertaining kids tv, a SpongeBench measuring entertainment value, etc. If a SpongeAgent spends 2000 years in Agent University learning how to Spongemaxx we probably don’t need or want to have it spend another 2000 years writing smoke tests
sorry but no, you deserve a pop in the mouth for dropping this one.
- Understanding of protocols like HTTP.
- HTML, JS, CSS, SVG, and everything "web".
- Understanding of databases, SQL, etc.
- Abstract code architecture patterns.
- Understanding the users' requests in English.
- Responding in English.
- Command line tool usage (agents/harnesses)
- Industry-specific knowledge that can be applied.
- Frameworks, SDKs, applicable libraries.
- Relevant legal requirements.
- Etc...
It would be trivial to have a pre-flight convo with an llm to guide the user thru module choices. "Build a site" -> "ok, describe the purpose" -> "local council in XYZ location" -> "that implies you won't need localization since XYZ has a monolingual government" -> "english and catalan localization please".
Right now, you prompt and it builds using assumptions, and we prompt to adjust. I think it would be great to be able to pre-load a set of assumptions.
Everyone assumes that carefully crafting a specific AI architecture with bits and pieces bolted together based on their human intuition is necessarily superior to simply using a bigger monolithic AI model. It turns out that the opposite is true, and has been demonstrated over and over again.
The bitter lesson is this: You can simply ask a frontier model to do the things you suggested, in a few terse lines of English. Dump a few lines in AGENTS.md and you are good to go.
Your approach is to "fiddle with inadequate tools" for weeks or months until you can finally attain a pale imitation of what the frontier models can do effortlessly.
It's the classic "But I can customise EMACS endlessly, why would I use an actual IDE?" argument all over.
I get it. You don't feel ownership over someone else's AI. You don't feel involved, you don't feel like you have agency.
It's like LEGO or IKEA furniture: study after study has shown that people enjoy things more if they "put it together themselves", even if fundamentally the thing is worse and/or still essentially nothing more than plastic made in a factory.
You don't _have_ ownership of someone else's ai, and that comes with real risks.
Security risks, privacy risks, business risk.
They might rug pull you, they might charge you more, or like atrophic, silently corrupt the answers, or code...
The labs are happy to jump on any emergent capability the scaling and training impart: generate prose, teach you things, cyber security, design, code, etc.
Do you really think that the frontier labs won't turn a popular capability, or trend they notice, into a first party tool if the ROI seems there? If it's your own private ai in your datacenter, you can keep it all secret, and not lose your business.
On the bitter lesson you're right of course:), wish I had a super computer to just scale that instead.
A) You can always self-host something like Kimi, DeepSeek, or GLM.
B) Just because you use a specific proprietary AI for programming doesn't actually bind you to that provider in any meaningful way. The authored code remains even if you stop paying them!
Of course, if you use AI as an active component in some sort of service, then the EULA, rug-pulls, etc... suddenly start to matter. That's a different story.
I think I agree with the bitter lesson, but I wish I weren't :)
> A) You can always self-host something like Kimi, DeepSeek, or GLM.
I mean, one could rent-a-box for, like, 10$$ per hour? Agentic loop development gets really expensive at scale with larger models, like, if you want to A/B test two tool schemas to see which works better, and you run 100 benchmarks...
But why are you so convinced the bitter lesson is true, and it's not just a temporary lead? Proper agent loops and RLVR are like, 3 years old at this point? At some point, right, the compute can't scale it out further? And at _that_ point the lead position might go back to: highest compute + smartest designed smarts.
Against my somewhat better judgement I'm currently "assembling small AI pieces" :(, for lack of access to unrestricted models for offensive security work and, ehh, funds. It's _okay_ so far, I'm running private benchmarks and look at the trajectories. To be perfectly honest, qwen is _really_ doing well, finishing quite complex chains without a lot of smarts in the prompt. Just "Go pwn {server}, use these {tools}".
But, there are some smarts embedded in those tools. Helpful errors, retries, benchmarked/handy representation. Strict validation of what the model tries to do, etc.
> The authored code remains even if you stop paying them!
This is true, but my point is that they will outcompete you if you happen to stumble on something that actually makes good money using LLMs and becomes popular. Obviously, if you don't then they won't.
> Of course, if you use AI as an active component in some sort of service, then the EULA, rug-pulls, etc... suddenly start to matter. That's a different story.
That's the plan hehe
That paragraph sets me off. I’ll take Vim and Emacs over VSCode and Eclipse any day.
> Visual Studio Code (commonly referred to as VS Code)[11] is an integrated development environment
Unless you were just going for a sick burn on vscode, in which case carry on :)
There’s no need to bring religion into this.
And of course the neural network series.
You just defined a liberal arts education.
Or yesterday, when I asked it for the best low resource approach to select the matching string from a collection of strings for a shell interface it suggested an exhaustive string compare over the entire collection. Because it was thinking of how David Lynch would tackle it.
I think you're missing the point of the article though, which suggests not to mix reasoning capacities with actual knowledge. Sure working with Swift and Python is basically the same, it's programming, with the same concepts etc. Much closer than the ability to drive a car. But the point is, the methods, libs etc are all different and things are changing each time a new version of the language is released. Like you don't need to relearn how to drive a car if you go to the UK but you have to known the wheel is on the right. Knowledge shouldn't be stored in the weights.
There are "experts" which do divide parts of the model that are found to activate together for specific tasks, so they can be processed in parallel to join the result at the end, but it's nowhere near the granularity of a SwiftUI expert and a python expert. The difference in those things is so trivial from an abstract point of view that it would make no sense. They would be 99% the same.
Distillations also come into this but I'm highly skeptical you could make one guaranteed to only know programming and only in one programming language (especially with as small a sample set as SwiftUI relative to something like C) without its efficacy being hobbled by tunnel vision. Reminiscent of the SpongeBob episode where he empties his mind of everything except fine dining and breathing, then can't remember his name and goes insane. Beyond the basic concepts of general coding and the trivia of syntax, getting anything done requires a large intersection of disparate world knowledge and the ability to apply it to new situations.