Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by TSiege on Tuesday April 21, 2026 @12:02PM
(#47850674)
latency absolutely matters? this is such a weird thing to say. for training sure, but customers absolutely want low latency
[ Parent
| on HN ]
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by electroly on Tuesday April 21, 2026 @12:05PM
(#47850731)
They want it, sure. Customers want everything if it's free, but this is about what they value with their money. In this thought experiment, you're Anthropic, not the customer. You're making a choice that's best for Anthropic. Will Anthropic lose customers because the latency is higher? No way. Customers want low cost and lots of usage more than they want low latency. In a cutthroat race to the bottom, there's no room to "give away" massively expensive freebies like a data center near every population center when the customer doesn't value those extras with actual money. It's the same reason we all tolerate the relatively slow batched token generation rate--the batching dramatically lowers the cost, and we need low cost inference more than we want fast generation. If the cost goes up we'll actually leave, for real.
After the initial announcement of "fast mode" in Claude Code, did you ever hear about anyone using it for real? I didn't. Vanishingly few people are willing to pay extra for faster inference.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by beisner on Tuesday April 21, 2026 @01:26PM
(#47851799)
They’re unwilling to pay for fast mode because of the current step function price increase once you hit your quota. It’s a psychological effect. Because most shops I know in the US currently paying $125/mo per seat for Claude would happily - HAPPILY - pay 2x, and begrudgingly pay 10x that amount for the same service. If fast mode was priced 25% or 50% more they’d happily pay for that too. But it’s just not priced that way currently with weird growth subsidization & psychology.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by CuriouslyC on Tuesday April 21, 2026 @12:51PM
(#47851348)
The only AI use case that cares about latency is interactive voice agents, where you ideally want <200ms response time, and 100ms of network latency kills that. For coding and batch job agents anything under 1s isn't going to matter to the user.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by electroly on Tuesday April 21, 2026 @01:05PM
(#47851512)
tbh, that's a good point about the voice agents that I hadn't considered. I guess there are some latency-sensitive inference workloads. Thanks for pointing that out.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by devolving-dev on Tuesday April 21, 2026 @01:45PM
(#47852006)
Yeah, also stuff like robotics which might not really exist today but could be big in the future.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by vanviegen on Wednesday April 22, 2026 @02:43AM
(#47859903)
You'll want the time-sensitive parts (motor control) to be running locally anyway.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by coredog64 on Tuesday April 21, 2026 @03:25PM
(#47853346)
A customer service chatbot can require more than one LLM call per response to the point that latency anywhere in the system starts to show up as a degraded end-user experience.
Re:Anthropic takes $5B from Amazon and pledges $100B in clou...
by blmarket on Tuesday April 21, 2026 @01:57PM
(#47852143)
Easy solution - use hyperscalers with super expensive API charge only when latency really matters. Otherwise build your own DC. Easy to expect customers don't care latency that much over money.