devonkelley

The 1 in 4 artists actually using the model for their own work is the most interesting data point here. If you're building a royalty system and 75% of the people being paid don't even want to use the tool themselves, that tells you something about the gap between "this is fair compensation" and "this is actually useful to my creative process." The royalty model might be the right thing ethically but it doesn't solve the adoption problem.

This is the right question and honestly the answer for most agent tools right now is "data gathering and context assembly, not generation." The drafting part gets all the demos but the real time save is the agent pulling the right info from 5 different sources before a human writes anything. Generation quality will catch up but the integration plumbing is where the value already exists today.

This is a really interesting edge case that's going to come up a lot more. If you feed an agent your test suite and it produces passing code, the tests shaped the output. That's closer to a spec than inspiration. The legal system is going to have a fun time drawing lines between "I described what I wanted" and "I provided a machine-readable contract for what the output had to be."

Taste isn't retroactive pattern matching though. In practice it's closer to "how fast can you tell if something is a dead end." The researchers and builders I've seen with actual taste don't pick winners, they kill losers faster. They have a feel for which problems have tractable surface area and which ones are going to eat 3 years of your life with nothing to show for it.

This is the core problem with every legacy infra company trying to wedge into the AI stack. They're pricing it like compute when the real value is in how fast you can iterate the hardware under shifting model architectures. By the time Oracle's DCs are online with Blackwells, the models will want something different. It's not a hosting contract, it's an arms race with a moving target.

The NFT/web3 comparison is tempting but I think it misses something. NFTs had no underlying utility beyond speculation. AI is already generating real revenue for companies that figured out the execution layer. The bubble isn't in the technology, it's in the gap between what gets announced and what actually gets deployed.

The circular investment thing is real though. Half the "AI investments" are just cloud compute commitments rebranded as strategic AI spending. That part is absolutely going to unwind.

That 42% number is telling but not for the reason most people cite it. The gap between "impressive demo" and "production system that actually saves money" is enormous. Most of those killed pilots weren't bad ideas, they were bad execution. No feedback loops, no way to measure if the AI was actually doing the job correctly at scale, and no plan for what happens when it starts failing silently.

The 10% that stuck probably just had better engineering discipline around it, not better AI.

The "non-deterministic microservices" framing is exactly right and I think most infra teams underestimate how much that changes things. With a normal service, you can map inputs to expected outputs and write tests. With agents, the blast radius is probabilistic and context-dependent.

The monitoring problem alone is closer to fraud detection than traditional APM. You're not looking for "is this thing up," you're looking for "is this thing subtly wrong in a way that compounds over the next 10 steps."

Honest answer: the problems start when you're running 50+ agents across 3 different model providers and the failure modes aren't "pod crashed" anymore. They're "model returned confidently wrong output and the next 4 steps ran on garbage."

K8s is great at keeping things alive. It's not built to reason about whether the thing that's alive is actually working correctly. Agent infra needs to handle rollback at the logic level, not just the container level.

Grammatically airtight and worth reading are different things though. The A/B test argument is real but it proves engagement, not quality. SEO shops winning doesn't mean readers are getting better writing, it means the bar moved to "good enough to not bounce."

The 20 minutes of distracted coffee-spilled human writing at least had a voice. The LLM draft is technically cleaner, sure. But "cleaner" is doing a lot of heavy lifting when the output reads like it was written by a committee of style guides.

Containers solved "it works on my machine" for deployment. We still don't have the equivalent for AI agents. Every agent runs in a slightly different configuration of model versions, API keys, prompt templates, and tool permissions and there's no standard way to snapshot that. Docker gave us reproducible deploys. Agents need something similar before they're production-grade infrastructure.

Memory is becoming the new compute bottleneck for AI workloads and Apple just accidentally confirmed it. The whole "run it locally" story for agents depends on having enough RAM to keep the model and its context window resident. Every GB of memory you don't have is a capability you can't run. This matters way more than clock speed right now.

Files won because they're the lowest common denominator that both humans and machines can read without negotiating a protocol first. Every agent framework that tries to replace files with some custom state format ends up reimplementing half of a filesystem anyway. The boring answer is usually the right one.

This is basically "write good specs and you get good output" which has been true of every engineering process in history. The LLM-specific version of the problem is that when the spec is ambiguous the model will confidently pick an interpretation and run with it instead of asking. Human juniors at least sometimes say "wait, which one did you mean?" That missing feedback loop is where most agent failures actually start.

This is the thing people miss when they debate whether AI coding tools are "good" or "bad." For a certain kind of builder the bottleneck was never the ideas, it was the implementation friction. If removing that friction puts someone back in the seat at midnight shipping things they care about, that's the whole point. The tools don't replace the taste, they just let you act on it faster.

The irony is that the same models generating spam Show HN posts are the ones people are building products with. The signal-to-noise problem on HN is just a microcosm of what's happening across the entire AI tooling ecosystem right now. Tons of wrappers, tons of noise, very few things that actually work when you put load on them.

The gap between "this model can answer my question" and "this model can reliably execute a multi-step task in a production loop" is where all the interesting problems are. Local models are getting surprisingly good at the first thing. The second thing is still a frontier model problem and honestly the frontier models barely do it either.

Karpathy's observation that models feel "cagy and scared" on open-ended problems is the most important thing in this whole thread. The RL training loop that makes these models useful for constrained tasks also makes them conservative when the problem space is ambiguous. That's a fundamentally different issue than capability. It's a disposition problem baked in at the training level.

25% regression rate on the best model is the number people should be sitting with here. That means 1 in 4 commits from your agent is breaking something that used to work. In any human team that would get you a serious conversation. We keep benchmarking agents like they're taking a test but the actual failure mode in production is slow accumulation of regressions nobody catches until the whole thing is on fire.

The funniest thing about this list is that it exists at all. We now have a cottage industry of people trying to sand the fingerprints off AI-generated text instead of just... writing the thing themselves. If your idea is good enough to share, it's good enough to spend 20 minutes actually writing. If it's not, a cleaner prompt isn't going to save it.

"This is a state machine, not infrastructure" is exactly right. Everyone overengineers this.

One gap: key health doesn't tell you model health. Your carousel can happily route to a "healthy" key on a model that's returning syntactically valid but semantically garbage output. The failures that actually hurt in prod look like 200 OKs.

Retrying the same model with the same params usually just gives you the same bad output again. And hardcoding a fallback order means you always eat the cost of your second choice even when the first one works 98% of the time. The smarter version is making that fallback decision dynamic based on what's actually working right now for that specific prompt shape.

Most observability tools in this space are dashcams. They show you what happened after you already got robbed.

The gap isn't monitoring. It's what happens automatically when degradation gets detected. Right now the answer for every team I've talked to is "page a human." That human reads logs, guesses, deploys a fix. The system already shifted while they were debugging.

Sandboxing solves "prevent the agent from doing damage." The failure mode it doesn't catch is when the agent operates perfectly within its permissions and still produces garbage because the model degraded or the tool stopped returning useful results.

That's a 200 OK the whole way down. "Prevent bad actions" and "detect wrong-but-permitted actions" are completely different problems.

Everyone in this thread is debating definitions. The only question that actually matters is economic: when does AI flip from "powerful automation with humans propping it up" to autonomous output?

Go look at any production AI deployment today. Humans still review, correct, supervise. AI handles volume, humans handle judgment. Judgment is the bottleneck. You haven't replaced labor. You've moved it.

Global labor comp is ~$50T/year. The entire capex cycle is a bet that AI captures a real fraction of that. Whether you call that threshold AGI or not is irrelevant. Capital markets don't care about your definition. They care about whether labor decouples from output.

Maybe, but "really smart" is doing a lot of work there. Even if the model is 10x better, you still need something to manage tool failure, retries, rate limits, cost budgets, human approval gates, and state persistence across long-running tasks. Those aren't intelligence problems, they're infrastructure problems. The framework isn't going away, it's just going to get a lot thinner.

Single agent with good context management beats a sloppy multi-agent setup every time, I agree. But you hit a wall around 100-120k tokens of accumulated context where the model starts losing the thread. It'll forget a constraint from early in the conversation or subtly contradict a decision it made 50 tool calls ago. That's when you need to split, not because the architecture is elegant, but because the model literally can't hold it all anymore. The question is whether you split into a pre-planned graph or let the agent decide when to split. Both have real tradeoffs.

The add/remove churn is the thing nobody talks about enough. We track net lines changed vs gross lines changed as a ratio and it's genuinely horrifying sometimes. An agent will touch 400 lines to net 30 lines of actual progress. And the worst part is the removed code was often fine, the agent just decided to refactor something it didn't need to touch while it was in there. Shorter, tighter loops with clearer scope boundaries have helped us more than any orchestration improvement.

The real issue with "just reviewing code" is that code review skill degrades when you stop writing. I've watched it happen. The people who are best at catching subtle bugs are the ones writing code daily. Pull them into pure review mode for 6 months and they start missing things, not because they got dumber, but because the pattern recognition atrophies without the feedback loop of writing, running, and debugging yourself.

The approval-link pattern for gating dangerous actions is something I keep coming back to as well, way more robust than trying to teach the agent what's "safe" vs not. How do you handle the case where the agent needs the result of the gated action to continue its chain? Does it block and wait, or does it park the whole task? The suspend/resume problem is where most of these setups get messy in my experience.