yetanotherjosh
But, "coding" is, except that solving it means using techniques that are still barely understood and barely described today (something I'm hoping to be able to communicate better myself about). It is now possible (at least for most consumer software, I would not say applications where human life or extreme risk is involved) to work exclusively in the domain of natural language requirements, natural language architectural/design decisions, and natural language test cases, and deliver product of equal or better software quality than average human code authors could have produced. Even in a language that you plausibly don't actually know how to code in, because the language itself can often be separable from the requirements and verification process.
If someone who doesn't know a language or framework can plausibly produce better software using it, and faster, than a team of people who do know the language/framework (and I would strongly argue that is more than plausible now with models like Astra and Fable) then it's not just "coding is less expensive."
It's like saying computers make complex calculations less expensive. That's true, but it's missing the real paradigm shift.
Natural language test cases still define the expectations both at the product and architectural level and are essential for triangulating the agents on successful outcomes.
A requirements and verification approach with agents is not waterfall any more than TDD is waterfall. Does thinking ahead and doing some planning equal waterfall? Does describing how a feature works to an end user, and making some key technical decisions, before you build it, mean waterfall? Does having some sense of what you're building first mean waterfall? With agents, you can specify (with PRD and technical specs) what you THINK it should do, and in minutes or hours or at most days, have the result, which you then learn from and iterate. If you didn't fully think it through, the agents will do one of three things: 1) decide for you, which you learn from 2) stop and ask, which you learn from 3) introduce bugs, which you learn from.
It's extremely iterative.
Just to make this clear: if you can define a really good PRD and sophisticated technical specs, and a strong set of tests cases to pass, at the right level of architectural granularity, plus adversarial code review processes that triangulate and weed out most mistakes, SOTA agents can write the code autonomously, at or above the quality level of most human coding teams. I call that "solved" but only if you meet those context requirements. Which is still hard, not solved, at that layer.
Solving writing is not a good analogy IMO. Writing is for human consumption, and cannot be wrapped in objective requirements and verification processes. Certain forms of writing perhaps could be (can't think of one at the moment but I don't doubt some exist), and those forms might be good analogies for being "solvable" or "solved."
I need an explainer. Each of these products is carving out a particular niche, or competing directly for someone else's niche with better X or Y, and I'd love to see some analysis of the landscape.
1. Model routing. Do you regret giving a task to Fable when it could have literally been done by Sonnet?
This is the one thing that I can understand.
2. Targeted code + context search. We think embedding the whole repo is dumb. We also think sticking the whole context (or compressed context) in chat is dumb. So we do faster and better greps over both.
Is this just a different system prompt? Why would other harnesses not be able to do it this way given instructions to do so?
3. Aggressive context hygiene. Tool output is bounded, stale screenshots disappear, we don’t re-read files…the garbage never floods the model.
Existing harnesses already don't re-read files that have not changed. And content that is already in the context window is cached tokens and doesn't add significant latency, what am I missing? Removing context from history invalidates KV cache.
4. Efficient turns. Batch independent investigation, make one surgical edit, then perform one focused verification. Internal measurement showed 16% fewer round trips and 27% lower cost.
Isn't this just a specific sub-agent strategy, also addressable with prompting?
I'm genuinely confused about what is mechanically different in this harness that could not be accomplished with a prompt/skill in another harness.
Or is your definition of "open source" mean that a small indie dev should be able to reproduce the entire pipeline? Because that would disqualify more than just LLMs, but also hardware platforms like Arduino where you need to pay for manufacturing to get the underlying stuff built... is Arduino "open source?"
"The current behavior paper" -> The behavior in the running system that was previously described as papered over.
"Marker transport over-claim" -> The inaccurate review finding on the object's sentinel flag in the API response.
I would gladly accept terms like "load-bearing" as they are reasonably clear and are not invented on the spot. It least communicates, even if you find the gravitation to that phrase annoying.
I suppose the cryptic/invented language problem is about token efficiency? But this sort of token efficiency is extremely difficult to deal with when it comes to conversation with a human about complex system. It might be efficient inside reasoning blocks, but when the model generates the final turn text, it should avoid this, as it's brutally inefficient due to the time spent wondering what each uniquely coined phrase means and having to ask for constant clarifications, which then you have to wait for another turn, eating up time and context while it burns more xhigh reasoning just thinking about how to explain its own awful language.
Codex is a harness, while GPT-5.5 is a model. The last codex-branded model was 5.3. Codex as a harness ships as a CLI, a desktop app, and a web product (and I'm not at all sure how similar the underlying harness is between them.)
Is the bug here supposed to be with the CLI harness, or the model? Does it also happen in pi, opencode, etc while running GPT-5.5?
To be fair I appreciate the effort of running and sharing the test. It will hopefully lead to better ones. But this is not a great test. Super interesting to think about what would constitute a better test.
For one, I think the agent would have to be expected to have productive interaction through the email channel, in a way the user depends on it generally working for some real world use case / value prop. In other words, needing emails to actually have the agent really do work, respond with results, etc. Also, most requests should be legit and the real attacks should be intelligently disguised, not pitiful/joke-level spam (although those would be arguably realistic to have in the stream, but, perhaps only as deflection so that the real attack is mischaracterized.)
> LongMemEval rewards recovering literal witnesses: exact dates, counts, preferences, and spans that often remain stable under tokenization.
Is this saying they chose a benchmark that is biased towards doing well against literal string matching, thus works well with grep, and then (gasp) showed that grep did well, finally declaring "grep is all you need"?
The examples in the benchmark's demo image(1) are all examples you could see grep doing well on. A conversation about bikes, then a query about bike(s) where "bike" is a common token hit. But not stuff like a conversation about a Beethoven sonata, then a question about classical music, where embedding based approach would shine.
(1) https://github.com/xiaowu0162/LongMemEval/blob/main/assets/l...
When I read that, I thought they must be using 'fork' wrong, and actually mean branch on the official repo, as that can't be right!?" Good lord.
My big takeaway from it was that breathing and neurological state are deeply connected and actually relaxed, natural, healthy breathing (and the corresponding state of the brain and nervous system) is something that most people have probably never even experienced unfortunately. We all think our state is normal, but I assure you, it is very far from the state where your control pause is 40s-60s or more, it's a radically different experience.
Also the nuance of what the control pause and how to measure it correctly is lost on I would say, even most people who attempt to learn Buteyko. The control pause is how long you can, under normal breathing, suspend breath with zero discomfort, and then prefectly resume normal breathing without any change from before. If you took a bigger breath to start, or when you start breathing again it's even slightly heavier, it's not a control pause measurement, it becomes an ego metric juiced to make you feel better about a number while avoiding the disappointing facts.
Buteyko claimed that healthy breathing had a 40+ second control pause. Which if you think about the real meaning and how to measure it, is a super long time. And I got there sometimes, it's a major learning experience about what deep alignment and relaxation of the brain/nerves can really feel like.
If you commit to nasal breathing for exercise as a constraint, it does force you to modulate your exertion while also increasing your CO2 and developing, according to the Buteyko folks, a new baseline for respiratory health and capability.
If you're running for your life from a tsunami, by all means mouth breathe. If your purpose is maximum exertion, of course mouth breathe. But that's not the only possible purpose of exercise. It can also be about respiratory training. Nasal breathing becomes a natural guideline/modulator for long term improvement in that regard.
It's like writing an article about "the dangers of PostgreSQL" ... when generating SQL from an LLM. It has nothing to do with Postgres specifically, it's that you're generating queries to run in a trusted context from an untrustable origin.
"including six dense models distilled from DeepSeek-R1 based on Llama and Qwen. "
people just don't read? not sure there's reason to criticize ollama here.
In the bigger picture, people use products to perform a function that often includes how they feel in and about themselves while using it. People buy cars to tell a story about who they are. People buy coffee and sit at a cafe in order to get a moment of peace or some internet access. People wear shoes that are a little less comfortable in order to get compliments on them. This is not bad UX. This is literally the meaning of good UX - paying attention to what people actually want versus the minutia of objective functionality.