Batch inference is much more efficient. Using the hardware round the clock is much more efficient. Cloud can absolutely pay more for hardware and still make money off you.

Cloud can pay more for RAM until all the RAM producers withdraw from the consumer market, then prices will go back down.

End users will still get access to RAM. The cloud terminal they purchase from Apple, Google, Samsung, or HP will have all the RAM it will ever need directly soldered onto it.

I was really fucking hoping we weren't at the part where "cloud terminals" doesn't seem farfetched and paranoid and yet here we are. Jesus Christ.

The next step, I think, will be a "cash for clunkers" program to permit people to trade in old computer hardware to the government—especially since operating systems that do not collect KYC data on their users will soon be illegal to operate.

Doesn’t Apple place RAM directly into the SoC package? We aren’t even talking about soldering it to mother boards anymore, it is coming in with the CPU like it would as a GPU.

Ram upgrades are happening because of ddr5. Nvme upgrades are happening because of pcie5. Prices will come down once everyone is done upgrading.

The hourly cost problem is worse for agents than single-model calls because context accumulates across steps. each tool result re-bills everything before it. Rate limits are a ceiling but the quadratic curve hits you before the ceiling does. We built Traeco to surface that curve at config time, not billing time. traeco.dev