If tokens are being given away below cost, of course you're gonna use more. It's not gonna last. How many tokens would you spend on random experiments if they were much more expensive? If tokens are so valuable, why do I (and many others) get to use them on economically worthless things until I'm bored?

We just don't know the true value of tokens until those subsidies stop, and the subsidies don't stop because the respective companies don't want to know, at least until everyone cashed out. The valuations of AI companies are very difficult to reconcile with current revenue (which doesn't cover cost) and with a sober mind it's hard to imagine where all this additional revenue is going to come from, down the line. And that's not to say the the technology isn't fundamentally transformative and revolutionary. So was the internet, but it still created an investment bubble.

The RAM prices are in yet another pork cycle, which will be exacerbated by the fact that many previous pork cycles left RAM manufacturers with excess capacity and rock-bottom prices, so they're much more careful investing and more assertive negotiating contracts now.

Tokens are not given below cost.

I'm regularly using thousands of dollars worth of tokens for a flat fee of $200. Their API margins may be good, but not that good. This is below cost. Therefore, if tokens are so scarce, so valuable, in such high demand, why are they still handing out such generous packages? Clearly there must be enough excess capacity to run this advertising campaign, and clearly the advertising campaign must be deemed necessary to maintain the belief in massive future demand growth.

85% gross margins.

does that take into account model development costs?

That is rather a big move of the goal posts. The previous post was that inference was unprofitable which is simply not true.

No, this whole thread has been a back and forth of pro AI(or at least pro anthropic) people arguing that the company is profitable because if you constrain your view to the last 2 quarters and just inference it looks that way while the other side keeps pointing out that you don’t get to do inference or the last two quarters without the massive cash spend occurring on training due to hardware and buildout.

These models become obsolete in months. Until the labs prove that they can produce a model that has a positive return on investment in it, it’s insane to think that the labs are in a long term good position.

Again, that is not what the person I was replying to said.

I didn't say inference through the API was unprofitable, I said token generation is heavily subsidized, artificially inflating demand.

Semianalysis estimates that subscriptions make up 40% of tokens generated for only the 10% of the revenue:

https://newsletter.semianalysis.com/p/anthropic-subscription...

I'm also not saying that this subsidy is irrational, it does buy mindshare, but it still distorts the true demand for tokens. I wouldn't be using anywhere near as many tokens if the price was as high as the API costs, if anything I would switch to a cheaper model, and likely not an Anthropic one.

If you're gonna point to gross margins on the API, those 85% really are meaningless in isolation. They can charge top dollar because they have one of the top models, but only because they keep spending top dollar on training it, as does OpenAI. It's a game of chicken where whoever stops heavy training investments first is likely the one to lose market share to whoever keeps spending. So the play is to try and keep the investment capital flowing and be the last one standing, which only works if you can keep selling a fantasy of future revenue, not the dire reality of current revenue.

If you'd read the article, or the sec fillings, you know that all inference from anthropoc is profitable.

If you'd carefully read the article, you'd notice a sentence like this:

"Sales and marketing includes compute for free access to the service"

It's safe to say that free inference is not profitable, but some back of the napkin calculations will show that getting an order of magnitude more tokens out of a subscription than the API price is not profitable either. Nobody else offers anything close to that much compute for the price, except OpenAI, which is running the same advertising strategy. If this was profitable, you'd have a a cottage industry of compute re-sellers selling plans for access to comparably-sized open models at similar token volume for a similar price.

i don't think it is. when you sell something you take into account the production cost associated. you don't have inference without the model production cost.

Goal post has been moving for a while on HN.

2023: Hallucinates too much to be useful.

2024: LLMs have plateaued. Scaling laws are dead.

2025: No one pays for AI

2026: Inference is unprofitable and unsustainable

For many people, if AI finds a cure to cancer, they'll say it's still useless because it hasn't solved immortality yet.

News flash, all four of those are still true in 2026.

If inference at Anthropic's pricing strategy is both profitable and sustainable, where is my $20 plan for similarly generous access to Kimi or GLM? It doesn't exist. Because it's not profitable.

AI is a bubble that will pop. No one really can say when Those both things can be true and second one doesn't invalidate the first one.