Microsoft adding Deepseek support already as I recall?
That is - for any definition of "they are behind X months" then eventually they get to the point Claude was in January when the world freaked out, but at 1/10th the cost. A lot of firms are going to mandate that is good enough for their developers.
I believe this hasn't been confirmed yet but I think it speaks to a bigger problem for the AI companies which is, if you give capable developers a good reasoning LLM, they can make it work like it was a really expensive model.
I believe we are 100% at the stage of good enough for the vast majority of tech companines. Fable and others will be more valuable for non-traditional tech companies.
I read somewhere that the chinese AI companies are sharing knowledge and it would not surprise me if the government is applying pressure by saying work together or else. If they work together, they can truly commoditize LLMs and with China ramping up hardware support for AI, I see the future being inference speed and hardware being the moat.
Which makes sense to me. Selling a chatbot interface/model access to the general public was never going to be a viable long term play. You still need developers to wrap the models into specialized tools. Queue the Jobs quote "It's a feature, not a product."
The only hiccup in that happening is will the US Gov let Anthropic and/or OpenAI fail when that time comes.
I built my career on Solaris and it got rugpulled by Linux.
That wasn’t because of software, it was because of hardware. Linux’s cost advantage existed because Sun hardware had huge margins, because their software was basically free.
AI will probably be a repeat of this. Whoever can come up with the hardware solution that minimizes the cost per token will win.
I believe the 5090 still holds this crown, but someone certainly knows better than I do.
And of course the C-suite will have unlimited access to Mythos tier models, which they'll use to summarize reports, while passing down mandates to rank and file to increase usage of less expensive models.
OpenRouter charges an extra 5.5%, Fireworks does not, Google is separate, but I doubt it will take 18 months. They are already aware they are losing business.
OpenRouter is the wrong abstraction for enterprise, we only need one model provider, not everyone in the world. Nor do we want to have to worry about failover going to providers we don't want.
But that's because I never got on the "run three dozen agents in a ralph loop" trend or other high-token usage methods. The way I use AI is discrete and targeted and it seems that's how it will be for everyone once the economics settle.
At least on a personal I feel like I’ve been getting the same amount of work done but I have to think harder rather than sitting back and prompting and waiting.
(We've received similar guidance. Even better is that our provider, GitHub Copilot, does not provide usage information to individual users if there is no per-user budget configured. So we just fire our requests into the black box and at the end of the month when IT gets the bill we maybe get a talking-to if it's excessive.)
I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because teams were building the same tools for what they thought was nothing.
Then they signed contracts with Anthropic and Google because I would assume they saw the token usage was through the roof. One month later? They completely cut off access to everybody for both Claude and Gemini. If you wanted access? Suddenly it was several forms, along with several approvals and a rock solid business case why you needed it. And before you got to the forms? You were added to a waiting list that was thousands of people long.
The entire company is now in damage control after trying to get the genie back in the bottle. I'm guessing someone saw how much we would be paying for the tokens we'd been using and decided to shut the party down so to speak.
Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would affirm or deny any of this had a positive impact. Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks.
What do you mean placebo effect? They thought things were created with AI while they actually weren’t?
So a lot of motion (do AI) was created without a destination (product outcome). The motion was what was being measured (we are doing AI).
For a large part of society we have management in leadership positions.
http://theinfosphere.org/Bender_Bending_Rodriguez#Compositio...
Isn't this a good thing? Doesn't it mean your company doesn't want employees to treat LLMs like a slot machines?
Trillions of CapEx for good Clippy/Siri, nice.
Institutions are neither permanent nor superintelligent
In my org, as our Team account (quota-based) was growing, to onboard more people we _had to_ switch to Enterprise (and thus API-based costs). The switch and now cost visibility really drives home the "did this code review worth $15?" kinds of questions...
We are just building the monitoring and I think there will be a crack-down sooner rather than later.
I'm more "code as craft" person, so I'm mainly unaffected (plus found my favourite local LLMs to be the missing rubber ducky with no associated cost), butI cannot say the same for everyone else around.
That’s what makes the AI Wild West so fascinating to me. Companies just threw money at it out of fear of falling behind.
Now, I think that fear has subsided at many companies.
I always thought it was incredibly questionable that we were able to slurp down AI tooling with no questions asked. Meanwhile, if we want a piece of software for some other non-AI purpose, the prospect of spending $50/month per employee is met with deep scrutiny or outright denial by default.
A couple months ago, I started using Claude for some tasks related to processing really messy data. Our company was making a big push for its use, and a co-worker gave me a fairly impressive demo of what he was able to do with this data in an hour or two.
Our company's Claude subscription defaulted to Opus 4.7 and we were encouraged to use it as much as we could.
I don't think I ever did more than 10 or 15 prompts in a day; I tend to do very detailed prompts and then spend 10-30 minutes playing around with the results and resulting code.
Today we get this:
> TL; DR: We're nearly at NNN Claude users and growing fast. Three quick asks: (1) default to Sonnet for everyday work; we have launched XXXXXXX to help with clarity, …
> …
> As we scale, let's be smart about how we work. Claude offers several models, and for everyday work — … — Sonnet is your go-to.
Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more.
Obviously this won’t be on an individual basis but some kind of unit.
Either way, with how much I see these companies cutting back I have no idea how the big AI companies are going to be profitable.
It's easy, just:
Make economically viable cold fusion
Make flying cars
Make a perpetual motion machine
Just because AGI is 3 simple letters doesn't implicitly mean we will live to see it.
Similar dynamic to how league might as well be candy land compared to dota2s checkers and star crafts chess.
If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage.
In reality many companies start complaining at their employees when they hit $0.1*X or less.
I think the problem is more for long lived projects and developers as regular employees. As you keep working on the product, you should be developing your own mental model of everything: business, tech stack, how the company approaches things. And that's where LLM value drops down a lot, or it should, otherwise there is a bigger problem with the developer. A huge LLM cost for a consultant is in my opinion a lot more justified than a huge recurring cost for someone who's been working on the same thing for at least 9+ months.
After arriving at a seemingly workable design it helps spot oversights I've made either directly or by providing a stand in for a person to walk though the logic with (unlike a real person it doesn't become annoyed or impatient). Then it magically materializes any and all boilerplate along with 90% of the solution near instantaneously.
Yeah it confidently makes some truly bizarre errors that no half competent person ever would. Failing to go over the output with a fine toothed comb is inevitably leaving landmines in place. But I agree with you that your own mental model is required - if you have that then you will be able to spot the pitfalls assuming you actually put in the effort to read the output and really think about it instead of skimming. (As an example an LLM recently materialized an nftables filter chain for me that would have taken me quit a while to figure out on my own but for no apparent reason omitted a single key detail that would have completely broken all networking if it had been used blindly.)
But did you do it well? It’s not a snack, but a question based on general observation that every is praising the process, but no comment on the outcome.
The future scapegoat, aka maintainer.
So scaling horizontally in different markets seems like an advantage if a product is already mature. Which is exactly what Anthropic and Open AI are doing because they want to put their tentacles in everything.
Exactly this. Unless the new features directly drive new users or new revenue from existing users, for many products iterating 2x faster does not mean 2x ROI.
Additionally, from anecdotes here, at work, and in my network.. a lot of the unlocked developer velocity is going to fun/frivolous/extra things. I think part of it is developers have their own features they want for themselves that are the easiest and most direct thing to deploy LLMs against, in the absence of good direction.
Yes it's cool you finally achieved 100% test coverage, or you wrote a new utility that makes your job easier, or cleared the 2 year old ticket that was 100 deep in your backlog, etc. But there were ROI reasons these things were not done previously.
That's already putting aside the fact that development going 3x faster doesn't increase end to end output by 3x because <100% of a SWEs job is development.
https://en.wiktionary.org/wiki/yak_shaving
And I'll have you know I'm a Grand Master Yak Shaver.
(I think they are being irrational, and that the mental model they have of AI costs -- "how much are we spending on tooling for this developer?" -- is going to shift over time to something more sensible, but those kinds of short-sighted companies are the ones that are having cost panics.)
What company pays developers $40000/month and are they hiring?
Based on such a basic mistake, I wouldn't trust the rest of your comment. Everything you listed feels like made up arbitrary excuses. Most of the overhead per employee is fixed, that doubling rule of thumb is assuming an employee with a salary closer to the median. At the salary you're claiming, the overhead shrinks to something like 30% on top of the base salary.
If anyone took your numbers seriously they would run into the weird conclusion that an employee paid $50k actually costs a company $300k. That's the type of math error you're doing here.
The mistake of 'h' instead of 'y' typo?
Overhead is not fixed per employee - things like taxes and benefits scale with salary, not to mention the cost of tooling (hardware, software, AI token budget, etc) tend to be higher for higher-compensated employees.
You definitely shouldn't plan your business based on my comment, but you'd be foolish to ignore it.
Meta, Google, Amazon, Apple, OpenAI, Anthropic, Netflix, Nvidia, to name a few
we need to start using that median instead of using salaries from the Magnificent 7.
... and that also improves the company's financials.
Otherwise it's all fluff.
I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train.
Now there's monitoring, reporting, alerting not just on overall cost but on "over-use" of best/priciest models based on total-or-percent tokens/dollars, etc. All of this comes with direct developer engagement & standardized management escalation for holding it wrong.
To me this customer behavior does not smell like a product you can 10x the pricing on to get profitable. We have exited the exploration phase and now ROI matters.