Planet scale infra often requires some novel ideas at the architectural layer (the engineers can input those) and then it’s mostly in-distribution C++ / Rust / Go; most of the hyperscalers open-source their stacks, for one. But for two, a ring buffer, look-aside cache, deterministic hash, b-tree, lsm-tree, etc. are all well-known patterns.
Hyperscaling is also often a very clear objective function; i need this code path to run in this many microseconds/nanoseconds, so i can hit the scale numbers I need. Claude/Sol can extract prod logs and build a representative micro-benchmark, and then hill-climb on it autonomously. Thats straight up the fairway for the training set, even if it often has a high bar for finishing and requires sophisticated Workflows or lots of tokens to explore the search space.
E2A: sorry sorry, typo, I meant microGPT. I can see why this would be confusing.
So despite its importance much of it is actually pretty in-distribution.
Personally, I'm more on the LLM code is pretty bad from a design point of view, particularly when much of the code is already LLM generated.
This has been the pro-"write with LLMs" argument the whole time. "If you're not getting good results, you're not prompting well. FAANG are doing x% of their development with AI."
Yet all of the companies that I can think of that build on planet scale infra and have gone all in on LLM development have seen a marked drop in quality and stability since adopting LLM driven development.
So yeah, I don't buy it. When I hear that engineers are writing their with LLMs and I see quality improve or stabilize, then maybe I will start to question whether its a prompting issue.
At this point if you can’t get the agent to write good code then either I) you are in a very specific niche (like Karpathy trying to write NanoGPT) that is extremely out-of-distribution, or II) skill issue, you need to learn how to prompt better.
It’s fine to have a skill gap! Just don’t delude yourself that the tools are bad and everyone claiming they are good is wrong.