Coding with LLMs in the summer of 2025 (an update)

Re:Coding with LLMs in the summer of 2025 (an update) by bee_rider on Sunday July 20, 2025 @02:50PM (#44628124)

It isn’t even that unreasonable for the AI companies to not be profitable at the moment (they are probably betting they can decrease costs before they run out of money, and want to offer people something like what the final experience will be). But it’s totally bizarre that people are comparing the cost of running locally to the current investor-subsidized remote costs.

Eventually, these things should get closer. Eventually the hosted solutions have to make money. Then we’ll see if the costs of securing everything and paying some tech company CEO’s wage are higher than the benefits of centrally locating the inference machines. I expect local running will win, but the future is a mystery.

[ Parent | on HN ]

Re:Coding with LLMs in the summer of 2025 (an update) by andyferris on Sunday July 20, 2025 @07:51PM (#44630507)

I think it’s the time slice problem.

Locally I need to pay for my GPU hardware 24x7. Some electricity but mostly going to be hardware cost at my scale (plus I have excess free energy to burn).

Remotely I probably use less than an hour of compute a day. And only workdays.

Combined with batching being computationally more efficient it’s hard to see anything other than local inference ALWAYS being 10x more expensive than data centre inference.

(Would hope and love to be proven wrong about this as it plays out - but that’s the way I see it now).

[ Parent | on HN ]