I think this should desactivate the moral high ground from which Anthropic is trying to speak. That they would want to make distillation orderly IMHO is fair, but to make it illegal is very rich from any AI frontier lab, really.

ahem.. it happens even today, you can use open weight models directly and even fine tune

> Ownership of content. As between you and OpenAI, and to the extent permitted by applicable law, you (a) retain your ownership rights in Input and (b) own the Output. We hereby assign to you all our right, title, and interest, if any, in and to Output.

https://openai.com/policies/terms-of-use/

> As between the parties and to the extent permitted by applicable law, Anthropic agrees that Customer (a) retains all rights to its Inputs, and (b) owns its Outputs. Anthropic disclaims any rights it receives to the Customer Content under these Terms. Subject to Customer’s compliance with these Terms, Anthropic hereby assigns to Customer its right, title and interest (if any) in and to Outputs.

https://www.anthropic.com/legal/commercial-terms

Obviously there is some bad behavior going on in the distillation scene with gray-market token resellers but that is "just" normal fraud.

> to the extent permitted by applicable law, you (a) retain your ownership rights in Input and (b) own the Output

If the argument is that the model itself is under copyright protection then "as permitted by applicable law" would be doing some heavy lifting. Assuming that were true, given that locally-run LLMs exist, what would be illegal: the distillation itself or the provision of service of the distilled model?

The moniker gives them an air of scientific, knowledgeable, tranquil, pro-social, pro bono work.-

Of course they are entitled to kill off a few mice, or pillage the commons to forward their "lab" work.-

The Atlantic argued this (rather well, IMO) a week or so ago - "There’s No Such Thing as an AI ‘Lab’" - https://www.theatlantic.com/technology/2026/09/stop-calling-...

But it's a black box! Nobody knows whats going on inside! It's all transformative! Sure...

Their open-weights competitors like Facebook can at least claim some kind of public benefit, but it's still just running a well-understood algorithm on dubiously obtained data with longer and longer runs, give or take some inconsequential architectural tweaks.

Anthropic's mechanistic interpretability work is the most "lab-like" of these, but it's still just secondary to selling subscriptions and fear-mongering for regulatory capture/investment/publicity.

But I get your meaning. What should they be called instead? AI Sausage Factories maybe (cue Upton Sinclair?)?

That's actually great? Slaughterhouses killing off the collective genius of humanity and grinding it into a bland paste for mass consumption.

I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct.

I love the "illicit distillation attacks" framing from the incumbents. There's nothing illicit. There's no attack. You just don't like it because it threatens your market position and business model.

Obviously it's not possible to run a company whose value is predicated on its IP that uploads said IP to a third party which might get access to it.

This could mean every potential serious customer would have no option but to seek alternatives to these online services.

I don't mean this as rhetoric, I did not think many people (except possibly those operating under government contracts, and 'normies' who don't know about these things) were under the belief that their IP was kept secret when they use these services.

https://privacy.claude.com/en/articles/10023548-how-long-do-... (see the additional info section)

You pay more for it, but if you care that much, use it.

The only verification you could do so far is GDPR-style data export, and also the adherence to GDPR regulations (and even those might get skirted if they aren't operating in europe).

AWS and Azure give you the same thing for Claude and ChatGPT, no need to be stuck with open weights. They might sometimes store some of it for other purposes (I don't know the specifics), but it is emphatically not being fed back to OpenAI or Anthropic.

Could you please clarify what that means? Given what I've been searching for, I might in principle be part of your intended customer profile, but I can't figure out whether you are merely doing routing (alternative to OpenRouter) or also inference (alternative to the names I've mentioned above). If it's merely routing, then how do you protect me from any potential misbehavior on the part of the inference provider?

Just feedback for what you're building, so please take this in a positive spirit... I'm an AI researcher and not quite an infra guy, and I'm making recommendations on token APIs for several less knowledgeable around me (I've gotten a few people set up with Baseten recently), and I couldn't figure out whether/why I would be interested in TrustedRouter. You should communicate the story better :-)

EDIT: Here's what I now understand after some digging; please correct if wrong.

There are some M token providers (not the names I listed above?) who provide cryptographic guarantees about inference services. But somebody still needs to verify what they do on each request. For an individual running a single harness, that harness would be a logical place to perform this verification if possible. For an org with N users each running their own harness, TrustedRouter solves the N*M problem and becomes the single gateway for trusted inference -- provided one somehow trusts/verifies TrustedRouter.

it’s all transparent and on github. i’d recommend just pointing your agent at trustedrouter.com since its well documented but quite a large product

The services have toggles to allow prompts to be used in the training set. There is a conspiracy theory that the toggle is a false distraction and they’re actually keeping everything, and that none of the employees involved will ever whistleblow this fact.

Outside of Internet comment sections, I think most people assume these US-based companies are doing what they say.

For enterprise use there are services like AWS Bedrock which have strict isolation guarantees. There are some people who still believe those guarantees are a lie, but once someone has reached that point I don’t think they trust anything that isn’t running entirely within their house. People in that category are a very small minority, but a very vocal minority.

It’s still a commonly held belief that “Facebook sells your data” and it’s cool to be cynical about everything tech in many social scenes. Conceding that a tech company might be honest about something will get you classified as a bootlicker depending on who you talk to so the only winning move is to be super cynical.

Among actual professionals I work with in tech and legal, almost nobody holds a belief that these companies are blatantly lying to their customers (and zero of their employees are whistleblowing it, while said companies also have employees trying to whistleblow AI safety on Twitter daily)

I don't think they are "blatantly" lying either, just normal bog-standard lying that we've all come to accept. It's a profitable and competitive tech company.

We have already seen this lying. The toggles are opt-out, not opt-in. When you sign up, you agree to binding arbitration, which is effective for preventing lawsuits in the US. The toggles are regularly turned back on without our consent on ChatGPT and Claude. OpenAI's "don't train on my content" setting isn't even in the ChatGPT interface.

As far as I know, they haven't suffered even a tiny controversy in public opinion over any of this at all.

There's nothing to whistleblow about when it's public knowledge.

How many of the people who checked those boxes have cryptographic proof they did it? How many of those people have opted out of the arbitration clause? How many of those people would be able to claim damages? Would the amount of people who satisfy all three questions be large enough to make it worth _not_ training on user data?

It’s truly trivial to “anonymize” and distill your prompts and model output. They could use just about any off-the-shelf cheap model for this. In fact, their TOS explicitly allows this, even with the toggle checked.

What that probably means is that the EXACT content of your prompt is secret. But the actual ideas are not. If you discover something truly novel, then yeah they get that. They can absorb trends in consumer behavior, too.

I’m sure if someone had access to all my paraphrased prompts, which retain 0% of my exact wording, they could find out literally everything about me. It’s a bit like how collecting metadata is as good (or better!) than collecting the real data.

And we all know “anonymizing” data doesn’t really exist like we think it does. Just removing names and identifiers doesn’t make anything anonymous for motivated actors. Or… say… an LLM that is trained to recognize patterns in text. Which is, like, all of them.

Your competitor is happy and continues to pay for the subscription. Sam and Dario just resold your code.

For what it's worth LLMs still suck at reproducing my little secret algorithm/implementation while being able to solve way harder problems. I have a good guess why that's the case.

You have access to all the frontier models, but...your inputs are not shared with the model vendors...neither are used to train the next model.

Why am I even doing the Amazon board job for them!??

I think this is being misunderstood. Codex has a toggle to allow your prompts to be included in training data. They’re saying they can’t be sure if the person had it on or off while using Codex to discuss the work.

They’re not saying that some prompts are mysteriously jumping into training data.

Also, there is a large market for AI services which don’t retain anything under any circumstances for enterprise customers.

[1] https://x.com/thsottiaux/status/2097746417012166816

Conspiratorial thinking is very common on these topics. Even bringing up the conspiracy theory about Instagram listening to your conversations and showing you related ads will bring up a surprising amount of people defending that idea on Hacker News.

https://www.wsj.com/tech/ai/jury-sides-with-openai-sam-altma...

Besides, true ZDR is usually offered by third-parties with deals to host OpenAI models, such as Amazon (AWS Bedrock) and Microsoft (Azure).

If you are big enough to be able to withstand that, you’re already running (or trying to run) your own/open-weight models.

2023:

"The approach also aligned with the company’s broader deployment strategy, to gradually release technologies into the world for people to get used to them. Some executives, including Altman, started to parrot the same line: OpenAI needed to get the “data flywheel” going."

https://www.theatlantic.com/technology/archive/2023/11/sam-a... https://archive.is/NmO5P#selection-979.907-979.1177

I don't think this has been a big secret.

LLMs should just pay a flat fee to use a specific book and thats it. Fees should be reasonable (not a million dollars per book), so long as the model doesnt spit out the entire book.

By your phrasing, it sounds like you still intend the possibility of companies owning copyrights; but how does that happen (other than copyrights already owned by companies grandfathered in)?

Copyright always starts off in the hands of individual human beings; it only ends up in the hands of companies when those human beings transfer ownership to a company. That ownership transfer can be automatic as a term of a contract, e.g. as part of a work-for-hire agreement. But no contract can cause the copyright to come into existence already held by the company instead of the individual. So if you abolish ownership transfer, you effectively make work-for-hire IP assignment invalid. What replaces it?

And, if "nothing"... then how do people pool the IP rights of their own small contributions to a large-scale work, into an IP pool that can be legally defended by a coherent legal entity, so that the large-scale work itself can have market value (i.e. so that sales of polished commercial bootlegs don't drive sales of the "authentic" work to zero)?

Keep in mind that, no matter how much we might want "mass distributed" media to have more-reasonable IP terms, the ability to sue for infringement is still critical to the existence of some forms of media. Especially "location-based" media, with no equivalent licensed broadcast right: movies still in theatre; concerts; live performances of plays and musicals; etc. If there's no legal team that can sue a movie theatre that shows an unlicensed copy of a given movie, then no movie theatre will ever bother with licensing movies again; "box office" goes to zero (from the movie company's perspective); and the incentive to create movies in the first place declines massively.

(You can see what this alternate world looks like from the few cases where movies screwed up the steps required to assert copyright, back before copyright was automatic. Night of the Living Dead (1968) is a good example: theatres — even upstanding large-chain theatres! — did indeed leap at the opportunity to show the movie unlicensed, and so Romero et al made effectively zero revenue off the work.)

I'm not saying this is an impossible problem. There are ways to accomplish this besides the way it's done now. (For example, individual-contributor IP could be retained by the original owners, but cross-licensed between individuals through a collaboration structure to form a coherent defensible IP pool, in exactly the same way that IP for e.g. video codecs is cross-licensed between corporations to form a coherent defensible IP pool today.) I'm just pointing out that the problem does need to be solved.

I would prefer it not be handed over across various copyright trolls, the fact nobody can sing "Happy Birthday" in a movie is still insane.

Or they could abide by the precedents they set and learn to compete. They shouldn't be allowed to have it both ways.

Vote (well-informed of the candidate's policies) in every election you can, even the local ones that seem of little consequence.

Convince others to vote.

Make demands of your elected representatives. You can mail them, call them, etc.

The government is the people.

The Reagan-era and beyond successful convincing of people that the government is an unchangeable black box made up of shady actors out to destroy everything (see: Republicans still going on about the 'deep state' when they run literally everything) is a big part of how we got to this place. It was a self-fulfilling lie, now coming true as the people who sold the lie start grasping for unending power.

But we still have the ability to vote our way out of it. If we continue to fail to do so, then at an evolutionary level we have to consider that we collectively deserve all the bad that comes from it.

See salvador allende in chile and mitternand in france for examples of voting without sufficient leverage.

your regan example occurred during a successful counterattack by capital that started under carter and crushed the labor movement.

Sometimes voting changes things, sometimes it doesn't. Naturally it's in the nature of most governments to not wildly change every election. This is usually thought of as an advantage. It's rare a wide majority of the electorate ever actually agrees on anything anyway, and tyranny of the majority or government that is too reactive to popular will is also not desirable.

But sometimes things are different, due to external or macro factors as you indicated. Some elections are more consequential than others. That doesn't necessarily mean any one election simply doesn't matter. It's simply good civic hygiene to vote. Think of all the people in red states who vote blue or vice versa in elections they will never win, never mind have their chosen candidate "change things". Does this mean they are fools? I salute them for such a faithful exercise of their civic privileges, that which generations of humans have died for. Your vote doesn't simply lose all value because you didn't get the outcome you wanted. It also doesn't preclude other avenues, like direct action or whatever your supposed alternative is.

It's good to vote simply out of civic obligation. Many people who say otherwise seem to merely be complaining that voting doesn't give anyone 100% of what they want. We've seen more and more "insurgent" candidates take over primaries. Increasingly people who complain about politics strike me as engineers who talk instead of build. Complaining about the effectiveness of your vote is like listening to an engineer who is always talking but never ships.

Like go run a campaign or solve the problem. What does saying voting doesn't matter solve? It feels like middle school age cynicism masquerading as insight. Politics has actually rarely felt so unsettled in America. Voting is likely more important than ever.

A politician who doesn't agitate voters to take direct action in between elections is ultimately one who hopes you really don't pay that much attention to what they are doing. It's always about electioneering or diverting attention elsewhere. It's fine to say voting is merely the bare minimum check on elected officials, as long as you don't encourage anyone to then skip this necessary (albeit preliminary) step.

Mamdani did do something interesting in this vein recently. When the racists on the right said he couldn't go to a 9/11 memorial, he released documents showing they allowed people to be poisoned in 2001.

It absolutely does change things even in those cases. You just voted for the wrong person if they aren't doing what their supporters want.

Voting changes things when those changes do not substantially threaten elites and they can come to some kind of acceptable deal, or when you have sufficient leverage that when your guy gets in they can steamroll the capitalists.

Leverage isn't "votes", that's just a preference on a piece of paper. Leverage is the ability to materially change reality to reward or punish actions. A strike, granting or witholding financial resources, shutting down infrastructure, armed revolt, etc.

It actually is leverage because those people who get the most votes are in charge of our government.

The government monopoly on violence, controlled by the people who get voted in, is the clear and obvious leverage.

> Mitterrand did attempt to do the things his voters wanted. He was constrained by capitalists who punished him mercilessly until he bent the knee.

Sounds like he didn't get enough of his own people in government then.

If you fail, then it means that you didn't get enough people voted in. Having leverage requires more than just winning a single election. It involves winning many elections and getting widespread support and your people in government.

People that control the government in principle control the guns. People that control the economy hold the real levers of power. You can in principle use guns to get control of the economy, though in practice it is trickier. America has done it on many occasions though, blasting apart democratic governments in South and Central America to install neoliberal puppets.

Thats not different than voting. If everyone flees, then that means that your agenda wasn't popular enough and you should get more support. The solution is still the same.

And the idea that just because you win 1 election somewhere in the world means that you get to infinitely enact your agenda is also silly. To fully enact an agenda you need widespread support.

And if your answer to that reality is to start asking question about how to prevent everyone from fleeing, then that almost definitionally means that your proposal don't have enough support.

So the point stands. Yes if you get enough votes you can enact your agenda. That says nothing about avoiding the consequences of your agenda though. You cannot use wish fulfilment and make believe to turn a bad policy into a good one. Yes, you have to suffer the consequences of your bad policies.

And it is not the fault of "The Elite", when people flee from the consequences of bad policy. Instead thats just the policy being enacted as expected.

Weird, weird weird take. How does any of this work. Like you cant influence the first decision but you can magically influence the second?

In 2028 a maximalist case could then be made to nationalize them.

The government actually running them isn't great, but no one ever gets 100% of they want in politics. It's about trafeoffs. If the labs use the current government to pull up the ladder, then the voters should vote for a government to swing hard to punish them.

In theory the threat of this should also act as a deterrent.

There's nothing weird about this at all. Politics is all about push and pull, carrot and stick. No one has permanent influence over affairs, and no one ever gets 100% of what they want. Yet it is entirely coherent to advocate for something like nationalization or take your pick of threat to private capital if they essentially cheat by pulling up the ladder because they've suddenly decided they don't want to compete.

Its very unlikely that AI will be on the ballot.

Why do you think AI will not be a major political issue in 2028? Perhaps to level set do you think it is a political issue now? Do you expect it to be less politicized in two years? Why?

Bernie wants there to be an extreme shutdown of AI immediately.

So lets say you had 2 billion dollars of slush and worked for OpenAI and anthropic.

You would immediately start to grease the paws of republicans to support just enough regulation to slow your competitors. And you would start to grease the paws of democrats to push for a "Moderate centrist" position away from Bernie, leading towards Regulation.

Ice Cream Vendor problem leads you straight into both sides supporting the same moderate regulation that OpenAI/Anthropic will write for them.

Keep in mind its going to be a noisy election cycle with Donald Trumps idiot successor vs whatever goofball the democrats vomit up arguing about tariffs, trade partners, healthcare, ICE etc.

Democrat falls out of line "You just want China to beat us at AI"

Republican falls out of line "You are just a techbro supporting epstein affiliate"

Then theres the standard 99% + chance regarding US election promises that people just dont act on it at all.

Etc etc etc.

However the frontier labs don’t have to serve customers who are farming the service for distillation purposes. That’s their choice and they’re free to make it if they detect distillation happening.

The only way US maintains dominance over Chinese models is by having an ecosystem of models. Relying on a small set of frontier labs will only let you get ahead temporarily. I agree with Gary Tan on this one.

I don't think a correct remedy is to require companies to provide services even if they want to. A simple example: you drop a client because their asks / ways-of-working / etc is more headache and costs than it's worth. I've done that before, multiple times, in my freelancing life.

(I assume there's a don't missing)

Forced open API access/protocol transparency for tech platforms and a right to client-side consumption, essentially making the server a common carrier would be one of the most elegant and simple ways to effectively destroy every issue internet platforms pose in terms of competitiveness or data ownership without creating endless fault-ridden legislation.

Tech giants aren't freelancers, they deal with billions of interchangeable users and aren't going to succumb to an annoying customer.

By copying their programming style, you'll move the model towards that way of writing, which will move the model towards the values expressed in those documents.

I feel that Deepseek v4 got so claudified at the end that it was like Claude.

Figure we’ll have to reckon with this next year in any case, guess we’ll see.

Someone with that lab could almost certainly figure out how do something stupid or destructive on their own, or bypass model safeguards somehow.

I’m sure the labs claim that their real innovation is in the RLHF, training and architecture. Keep that and just share the raw data somewhere.

Daily reminder that piracy is the only enduring archive mechanism.

Also, permitted or no, they are definitely torrenting. I would be deeply surprised if they hadn't already leeched every torrent on public trackers. The only reason they (probably) haven't depleted all the private trackers too is that they would be required to actually contribute back, which as above is never going to happen.

They were torrenting, they were sent to court and settled for big $$$$. The only other method available to them now is scanning, and scanning at scale requires the books destruction.

I agree that they should definitely be required to see the new scans but that would just be more $$$$ they get charged if caught.

I also think if Anthropic and OpenAI had been releasing Open models along the way, people wouldn't be nearly as suspicious of them.

Anthropic has never released anything but FUD.

> He also notes that the proprietary AI labs didn’t ask permission when they vacuumed up as much human knowledge as they could to train their models. They famously ingested plenty of copyrighted material without the permission of those intellectual property holders.

And many people's shared opinion (B):

>> I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct.

...

It is very difficult to actually say NO to the fact that (A) was done, which then leads logically to conclusions as (B). But also we should remember that if these two hold (and (A) is an axiom more or less now), then it comes as no surprise that then also all opensource licensing is immediately rendered void and null, as keeping it would contradict (A) and would go against the very common and consequential logic in (B).

Copyright is so dead. And it was not me killing it with a cynical post on HN. Dunno why so many people still fail to face it. There is no way it can exist in its current form, because then immediately (A) happens and (B) follows.

With what knowledge are you claiming this? If it turns out companies are using IP proxy networks would you change your mind?

What if the IP Proxy networks were used by criminals for similar attacks like DDoS or plain cyber attacks?

What if the source of the IP proxy networks were residential addresses to avoid detection?

What if the way these IPs were acquired were through pwned devices?

What if the credit cards used do not identify the company that carries the attack? What if they use the employee's personal credit cards? What if it's family members of employees? What if it's a network of personal credit cards where cc owners get a payment for making a purchase on their name? What if they are stolen ccs?

Not just a hypothetical btw, I believe almost all of these are true.

There is sooo much irony here.

Thats literally what is happening right now though. People are spending hundreds of millions on a training run, and then people are distilling them, fairly easily, and making cost competitive models.

We are seeing all of this in action right now.

So why is their own ToS so special? :)

I think a lot of this is done using stolen black market API credentials, which is why it might be somewhat accurate to consider it illicit.

Similarly, AI companies should be required to allow distillation at a fair price. Fair Use doesn’t make sense as a social contract if it only cuts one way!

Preventing token-consumers from developing competing products should be litigated as anti-competitive behavior.

Garry Tan and Sam Altman recently did this interview together. They seemed pretty friendly with each other during it. Wonder what Sam Altman would say about Tan advocating for OpenAI’s models to be distilled.

Then again this is the same OpenAI that has gotten into legal trouble recently regarding Apple’s IP so who knows

They know it won't happen, so arguing for it is 'effectively free' and purely personal marketing.

A bullshit game played by politicians and wannabes.

I think the only way forward is wealth tax. Rich accumulated so much wealth already, that they don't need to put it to profitable businesses.

it works for RE already without liquidation

> . I would opt for a tax on loans instead (you have to prepay tax on loans that use investments as collateral and no more step up in basis on death).

rich just will move from this scheme to something else, and nothing changes.

Maybe if we had a world government or something.

Also, I think it is equilibrium of interests of all involved parties: rich pay 0.5%/y of wealth tax, population is healthier, country is stronger and protects interests and safety of rich. The problem is mostly to make them realize this.

The US is increasingly a less relevant country every year. With a wealth tax, yes, all those Americans will have their wealth eventually confiscated (0.5% a year will add up quickly year after year, investments will unwind so wealth will stop accumulated, and eventually it's gone), but I doubt the rest of the world will care and just move on and decide crazy things that happen in the US are like the crazy things that happen in Iran.

US already have 0.5-1.5% property tax, no one calls it confiscation.

Markets will simply adjust and financial services will arise to smooth instances where liquidation is problematic.

The status quo shifts to optimize once taxes change. Our system will do the same.

I don’t understand why people don’t think it through all the way to the next equilibrium

The market will address by the wealth moving to where the wealth tax isn't.

Then the countries will start creating ridiculous exemptions and they become nothing burgers (e.g. see Switzerland's wealth tax).

Sure, we all agree.

The solution is pretty obvious, just implement a wealth tax that prevents capital flight.

It’s not that hard. The US already taxes its citizens abroad. A well-implemented wealth tax would be charged regardless of where you live or where your money is. If you are a US citizen you get taxed. Then you slap an exit tax on citizenship revocation and you’re good to go.

Not hard, simple, we already have systems in place to enforce it.

Back when people made arguments for software privacy, the argument was usually "big business will still pay and consumers wouldn't have paid anyways so it's ok for us to pirate" - I actually think that was fine for business software but terrible for indie games, whose market was 0% businesses.

But in the AI case, it's not like they get to keep some of the value of their investment - it all gets cloned into models that businesses and consumers alike are happy to use. If someone knows how labs could continue to fund data creation and acquisition in this model, please do share!

Not sure you get to count breaking the law and getting in trouble in your cost-of-doing-business. That's a little too on the nose.

You're basically arguing that a criminal syndicate must be allowed to continue and we're required to make their business model make sense?

This is how Uber worked. They didn't just break the law in different countries, in several they actively misled government/law enforcement investigations. Google "Greyball".

I would love for a US lab to be at or near the frontier with an open weight model, but it’s going to take some serious elbow grease, and yes some distillation (which btw OAI, anthropic et al, also use distillation of other’s outputs in their training)

It's also clear that, as Tan indicates, open-weight models will be (and basically already are) just as good as frontier models. It's all about the harness, baby. We will have two main forks in the road, and two new industries created:

    - AI hardware (NVidia/Cerebras/etc.), the equivalent of Intel/AMD
    - AI software (harnesses, assistants, etc.) the equivalent of Microsoft/Apple

Inference is not being subsidized and in fact has pretty high margins.

Similar-sized open weight models on openrouter are 15x cheaper per token than the big labs. This should reflect the isolated cost of inference, since 3rd party hosts have no reason to subsidize and no training costs to amortize.

Only datacenter buildout costs are being subsidized.

If it were true that Anthropic and OpenAI were profitable on all inference they wouldn’t need to constantly raise so much money. Anthropic regularly announce huge investments in infrastructure but it is all smoke and mirrors, data center build out costs aren’t being paid by OpenAI and Anthropic, they’re financed externally. Google, for example, are backstopping tens of billions of datacenter build outs that are being financed based on commitments but not investment from Anthropic.

You are underestimating the insanity of subscription subsidization. Being profitable on API inference is meaningless when it is such a small proportion of usage and is only going to fall off a cliff as cheap open weight models become more capable.

https://hraness.com/writing/my-girlfriend-asked-me-why-i-hav...

The absolute majority of tokens are being subsidized and as soon as the subsidies end usage will fall off a cliff, rendering all the data center buildout a terrible waste of money.

Do we know that? As I understand it, enterprise customers pay more. Do we know the usage breakdown between monthly subscribers vs enterprise accounts? I agree that it's inevitable that subsidized subscriptions are unlikely to last forever, but that's not the only assumption in your argument.

Edit: I think "enterprise customers pay more" was poorly phrased. I mean that enterprise customers are charged per token, presumably with a profit margin, and thus are not subsidized. While personal accounts are (thought to be) highly subsidized if you consistently max out the quotas. We also don't know what proportion of personal accounts do that though, which is another big question mark.

I know people that have the most expensive plan on all the platforms... because

The other side to that is, what is 'cost'? Is cost just inference or are expenses also being taken into account? Because the expenses of these companies are huge to build the models.

This is untrue.

You are way underestimating enterprise usage here.

You can't get the $200 subscription on Teams plans at all, and Enterprise plans don't have any subsidized plans.

Anthropic has 80% margins on inference: https://archive.is/BtEeN#selection-1575.0-1575.75

If $7.5bn is their enterprise revenue and it costs just $1.5bn to generate, that leaves $3.5bn in compute costs to account for. Dario previously said that training costs less than inference so training can’t explain it.

If subscriptions aren’t the majority of usage and aren’t subsidized, where is the money going? Anthropic don’t spend money on data centre build out so that can’t be it either.

I don't think you can reverse this out like this because the 80% rate is before payments to "distribution partners, including Amazon". I think that payment includes the hosting cost for that those model but it's unclear.

> Dario previously said that training costs less than inference

Do you have a source for that?

Are you sure you aren't conflating the statements Dario has made that training costs less than they make on inference (over the life cycle of a model)?

The "hosting cost" is paid for by Anthropic and is the largest cost. The money Anthropic pay to Amazon for delivering Anthropic models via Bedrock is separate, independent of compute costs, best thought of as commission.

The forecasted / guessed / estimated 80% number is based what customers pay per token minus the projected compute costs, i.e: the people who believe that Anthropic has 80% margins on tokens believe that Anthropic spend $0.20 on inference compute for every $1 of per-token billed-via-the-api revenue.

We know that there are hundreds of thousands of fixed-price subscriptions being used to their absolute maximum, with many people bragging about how many subscriptions they run in parallel. These tokens are not included in the 80% margins, they are acknowledged to be "subsidized". People like @theo on Twitter post almost daily about how much they're milking Anthropic and OpenAI with leaderboards.

Both Anthropic and OpenAI (more so OpenAI) do "resets" where they increase the limits available to people on their fixed price plans. We know that there are people paying $1,000 per month for multiple subscriptions to generate tokens that would cost $50,000 via the API. Even if Anthropic's margins are 80% on compute for per-token billing, that's still $10,000 of cost to Anthropic generating just $1,000 in revenue. Multiply that by tens of thousands or maybe even hundreds of thousands of subscriptions.

Anthropic and OpenAI have raised over $100 billion each and continue to raise. If they're making 80% or even 50% margins on $10 billion in revenue per month they would not need to raise, they would be shouting for the roof tops about how profitable they are, they wouldn't be delaying their IPOs, yet they're only profitable by non-GAAP metrics like WeWork's classic "Community-adjusted EBITDA" or in this case "per-token-adjusted EBITDA" or whatever they will call it in their IPOs.

Yes, they're selling tokens via the API for more than they cost, they are profitable on per-token billed inference, it has positive margins, but those profits are obliterated when you account for all the inference they're paying for out of pocket on fixed price subscriptions, upon which they keep increasing limits because they desperately need to show growth further harming their profitability (consuming all of the money they make from their API).

If Anthropic and OpenAI needed to be profitable tomorrow, they could be, they could kill off all their fixed price subscription plans and charge only for usage via the API, they'd print money, but they'd lose mindshare because nobody except for enterprises can afford to pay the true cost, all the regular people would switch to cost effective good-enough models, and then within months, the enterprises would start to switch too because no longer would their employees be claude-pilled.

Anthropic and OpenAI cannot turn off subsidization, thus, their margins on per-token API billing are not important in any discussion about their long term financial wellbeing. Just look at the large scale customers like Harvey (~15 trillion tokens per month, ~$50m+ in spend) who are, sensibly, investing in building their own specialized models that are cheap to run so they can cut their spend by 90%. That's profitable revenue for Anthropic / OpenAI today, but completely gone soon.

> Do you have a source for that?

https://www.youtube.com/watch?v=7xij6SoCClI

"This week, Noah Smith and Erik Torenberg are joined by Dario Amodei, CEO and Co-founder of Anthropic. Dario talks about the economics of AI development, the comparative advantage of AI companies like Anthropic, AI safety, and his stance on California's SB 1047 bill. They also discuss the impacts of AI on global power dynamics, competition between the US and China, and inequality in an AI-powered world."

At around 12 minutes in:

"I think actually even if such a model is released one thing you know that's a this analogy to to open- Source software is that these big models they're actually very expensive to run on inference the majority of the cost is is inference not necessarily the training of the model so if you have only you know I don't know 10 20% 30% better way to do inference that can kind of negate the effect so the economics are kind of strange yes there's this giant fixed cost that you have to amortise but then there's also the per unit cost of inference and small differences in that can actually again assuming the thing is deployed widely enough make a very big difference so I don't know quite how that's going to play out"

The scales have changed since then with inference costs falling and more being spent on training but the fundamentals are the same. Inference is expensive, in part, because peak usage dictates capacity whereas capacity can dictate training. Anthropic must pay billions of dollars per month to be able to handle peak inference, hence their efforts to try and shape usage by offering discounts / flexible limits at different times of the day. They can train when capacity permits.

My point is that you can't reverse out the maths like you did without knowing how much this is.

> These tokens are not included in the 80% margins, they are acknowledged to be "subsidized".

No, this isn't correct. Even including these they are claiming 80% margins.

There's nothing at all that indicates subscriptions aren't included in this - it's a simple statement of their running margin.

> these big models they're actually very expensive to run on inference the majority of the cost is is inference not necessarily the training of the model

I don't think you can take this statement to claim that currently they spend more on inference than on training. I think he's saying over the lifetime of a model maybe inference ends up costing more unless they keep finding "better way to do inference that can kind of negate the effect".

> If Anthropic and OpenAI needed to be profitable tomorrow, they could be, they could kill off all their fixed price subscription plans and charge only for usage via the API, they'd print money

My point is that their subscription costs are a lot less than you think because of this statement by Anthropic that they have 80% margins including these subscriptions.

If their margins are 80%, that means on $10 billion in revenue they're spending just $2 billion. Anthropic's own announcements put their spending at much, much higher, such as the $1.25 billion per month they are paying to SpaceX for compute, and the ~$3.5 billion they're paying to Google each month, and the billions to Amazon each month too.

https://www.anthropic.com/news/higher-limits-spacex

> We’ve signed an agreement with SpaceX to use all of the compute capacity at their Colossus 1 data center. This gives us access to more than 300 megawatts of new capacity (over 220,000 NVIDIA GPUs) within the month. This additional capacity will directly improve capacity for Claude Pro and Claude Max subscribers.

There's no world in which Anthropic has 80% margins. At their current expenditure on compute it would require at least $20 billion in revenue to be mathematically possible. The 80% figure on compute margins that is widely discussed is based on analysis by SemiAnalysis and refers only to their per-token compute margin (which is calculated comparing hardware costs + electricity costs to what they charge via the API).

The estimated training costs for models like Opus and Astra are ~$1 billion and they're not training multiple frontier models in parallel every month. Training costs cannot explain where billions of dollars per month are disappearing if they have 80% margins. And that's before even considering all the money they're raising and spending. Anthropic raised tens of billions just a few months ago, OpenAI even more.

Where is the money going?

No.

To quote from my link above:

> Anthropic has told its backers it will be profitable this quarter, as it moves to allay investor concerns about the aggressive cash burn of frontier AI companies ahead of its blockbuster initial public offering.

> The company has told a small group of shareholders that its adjusted operating income will be positive for the second consecutive quarter,

[snip]

> Anthropic’s gross margins are above 80 per cent before accounting for revenue shared with distribution partners, including Amazon, and the cost of training its models, according to two of the people.

So this is a direct claim by Anthropic people, not analysis by outsiders.

> If their margins are 80%, that means on $10 billion in revenue they're spending just $2 billion.

Agree - but only spending $2B on inference.

> Anthropic's own announcements put their spending at much, much higher, such as the $1.25 billion per month they are paying to SpaceX for compute, and the ~$3.5 billion they're paying to Google each month, and the billions to Amazon each month too.

I don't believe this compute is sorely dedicated to inference. Only the part dedicated to inference is included in this profit rate calculation.

I was referring to the "AI labs" here. Sam Altman himself conceded that OpenAI is losing money on the $200 subscription. Using open-weight/open-source models is indeed cheaper (and no reason for inference to be subsidized).

>Sam Altman himself conceded that OpenAI is losing money on the $200 subscription.

They have since stopped offering the $200 subscription, probably for this reason.

Subscription margins are harder to judge because it depends on usage; token costs are a better comparison.

1. Companies are trying to decrease costs, not increase it, and are looking at alternatives

2. Competitors are catching up, and even if the frontier labs are "better" at some things (like writing plans or complicated analysis), the competitors can take a lot of the inference on routine tasks like implementing a well-defined plan

3. The frontier labs don't just need to have high margins right now. They have to pay back their massive liabilities.

The US can no longer keep global trade secure on the high seas. What if the supply chains for GPUs get disrupted for months, a year? Then what?

I fear Google will win in the longer run.

Maybe it is “sunked cost” or maybe it is “I will do it myself dammit”.

    > inference is already being subsidized

the only place where actual subsidization (below cost) might be happening are the subscriptions. even that is unlikely because to be truly below cost you either need to offer below cost of electricity which isn't happening, have potential API users using multiple subscriptions or have opportunity cost loss due to saturation.

I do not agree with this man all that often, but that is very concisely put.

Well yes, as I think I said in a previous comment, on the current trajectory OpenAI and Anthropic will really stop releasing models due to distillation and regulatory pressures. Then, they would eat all knowledge work themselves, which would be the end of YC.

Following news of companies and projects increasingly moving to open weights models.

As AI gets more central to society, we really need to know how the weights were determined.

Open weights isn't just "free as in beer"; it can be "free as in the mystery drug that creepy guy chatting you up at the bar offered you". And maybe even he doesn't even know everything that went into the tablets, since he too was being worked, by an organ-theft ring who will be harvesting both of you tonight.

That's an analogy to get your attention. Your LLM probably isn't going to steal your organs. But in the current environment, it does and will have ideological biases determined by those with direct and indirect influence over it. And there will be a massive market for commercial influence biases (look at how previous generations of adtech invaded almost all technology companies). And there's incentive for military and spying capabilities to be buried in the models, perhaps as long-term sleepers. Maybe some organized crime trojans, too, depending which model you pick up.

In this low-trust environment of the current real world, we need genuine open source models, not closed "open weights", and not mindlessly distilling black boxes gifted by sketchy powerful interests.

Ban data brokers before you ban distillation.

Isn't this exactly what Dario wanted? He thought he knew what's best for the humanity...

At the end of the day they were built from data that did not belong to them. So it would be fair that humanity REQUIRES to give back the output of that.

It's a bit like the free software thing: you can still make money from it and providing service to it, but if you build it based on another free stuff the derivative should be free.

Why not do the same for intelligence ?

I suspect the top labs will come up with a business model that doesn't involve handing out their secret sauce for everyone else to reverse engineer. Perhaps restricting their top models to select high paying government/enterprise contracts. Or maybe a bespoke "describe the problem and we'll solve it for you" type service.

OpenAI doesn’t even know how their model works. Nobody knows how LLMs work. So it’s not like it’s technically difficult to replicate, just costly.

Same thing as Cliff Notes imo. In every other area of manufacturering and tech I can use a machine to build a new machine that competes with the original machine. Should Milwaukee be able to prevent DeWalt from using their drill to make a competing drill? Should Jetbrains ban Eclipse contributors from using their IDE?

It’s not like Anthropic or OpenAI have the faintest idea how their models actually work.

I think that software execs should not incentivize users or other execs to break Terms of Services, or contracts of any kind.

An executive or manager of a company that breaks contracts is worth 0, there's no incentive to do business with them, if you know they will agree to doing or not doing something and then breaking that promise.

The word of a businessman is their most valuable asset, Tan is signalling that he is either misinformed on what Chinese distillation consists of, or that it's ok to do it.

FAQ:

- "But the frontier models do bad things too"

- An argument worthy of a 5 year old, one civil issue doesn't negate the other, bring it to a court if you have an actual claim against OAI or Claude, etc...

- "Companies have the right to reverse engineer"

- Ok, do it, but the moment you are creating 10K accounts in a Distributed fashion (Distributed as in the first D of DDoS), using IP proxies and stolen credit cards or your employees and employee family credit cards, you are not doing it because you believe you have a right, you are doing it despite not having a right to it.

EDIT:

Re(actually)reading the article, Tan's take is a bit more nuanced, he seems to be advocating for regulation to restrict the capacity of Foundation models to restrict usage, on the basis (or to the extent) that it was trained on public data, and therefore it belongs or attributes its success to a wealth of the commons.

My pre-existing quip is against those that want to solve this as-is by breaking the ToS. I think that's a weak version of Free Software position, it's very weak to complain that some software is proprietary and want to use it anyway, the strong FS position is that you don't even want to use it if it's proprietary, you won't catch a FS activist pirating proprietary software, they just don't use it and develop alternatives. Similarly it's not a FS position to distill a proprietary model (where you still wouldn't have source code at any rate).

Genuinely fucking crazy we pay money for fast access to autocomplete of stolen human remains.

The fact that billions were spent on research to distill the internet should not preclude others from spending 10s of thousands to do the same to these frontier labs.

Time to create a bigger moat than "but we spent so much money doing this ...".

Those market entrants would face commodity pricing power vs. high capital costs, no?

Maybe there'd be ROI but I think there's another layer or competitive dimension that's neither frontier lab nor distilled model lab.

It's a good idea. It's not their data to begin with, anyway.

- theft / distilling (ie cheaper open models that are not from the soon to IPO US corps cheated rather than innovated)

- danger / nat security (ie we can only trust the soon to IPO US corps to shepherd us)