Let's put it this way, how much is 5% productivity bump worth to you?

If you're in the US and you're making 100k a year, that's worth 5k or $416/m. So you can buy two of the most expensive plans on the frontier models.

This focus on cost optimization is insane. Just use the frontier models. Even a marginal bump is worth whatever the hell they're charging, at least for now.

The problem is it might be worth it to the company, but likely not to you - a 5% productivity bump likely results in $100k a year.

You really think there's zero correlation between productivity and wages? Sure, it's noisy and you might stay at $100k or even get fired. But I'd say the expected wage value of 5% higher productivity on a large sample is at least 3.5% or so.

Companies aren't paying for tokens so that their employees can capture the gains.

The one man and the dog at the Carolina spinning factory earn far more than their equivalents in Birmingham.

Everyone else will be 5% more productive. Then no one is "more" productive. So everyone has a higher output, but the same wages and hours worked. There was only a gap when usable AI first came out, some contractors could do the same quantity of work in less time and enjoy time off or do more jobs. Now the gap has closed or is closing. And using AI now is more about not being less productive than peers who do use it.

That's not how productivity works. It's not a zero-sum game.

If all construction workers can build houses 5% more efficiently, that's not the same as nothing changing. Depending on supply and demand, it means 5% more houses are built, or houses are 5% cheaper, or maybe 5% bigger, or some combination. Whether or not the construction workers all get a raise or 5% get fired (or both) depends on that supply and demand, but historically they often get a piece of the growing economic pie.

Why would the company pay more when they can just not pay more? The only things I can see happening is they might lower prices as competition ramps up, or in general as there is more supply for the same cost.

If there's sufficient demand, that's just what happens.

To try and explain one path: Company A doesn't raise wages but makes 5% more money. Company B pivots from Industry B into construction (because suddenly construction is having 5% fatter margins), and hires workers at more competitive wages to poach them from Company A. Company A forces to raise wages.

If there's a demand ceiling on housing it's a different story though.

More like company B purchases a construction company and changes nothing but number go up for shareholders while wages stay stagnant for people producing actual value, as they have for decades.

If labour supply is fixed and productivity goes up then the value and demand for labour goes up, driving up wages

Why would demand for labor rise if productivity rises?

See the increase in CEO wages vs the increase in worker wages over the last 20 years of you want to know where that 5% will almost always go.

"Historically" is doing a lot of work here. It's a well-known fact that while productivity kept increasing, wages have stagnated since the 70s.

So you're right in an academic sense, but not for any practical purpose whatsoever, in terms of how it'll affect economy now.

Or, with current US policy, the top 10% will get 6% richer and you'll get 1% poorer. Sure, the pie can grow, but you won't (unless you are in the top 10%) get any of it.

large companies aren’t buying subscription plans. my org has a 2k per month token budget per person and starting to explore optimizations like automatic model routing.

There is no evidence that these tools provide a 5% bump, if anything they are providing a 20% liability (pulling random numbers is fun).

Also where is the evidence that the workers have ever benefited from productivity bumps? The only thing that happens is surplus gets captured by the owners while workers are forced to do more.

Bad deal all around.

You're saying this like I would see that 5k in my bank account. If I'm 5% more productive that probably wouldn't even make it into annual review, let alone pay.

> I wonder if we are at a point where the focus can shift to improving the cost of inference.

There's always working on improving the cost of inference, but I don't think this is an area of R&D that will slow down. The reason is:

1. A better competitor model risks eating away at how much they can charge for inference (i.e. revenue) 2. Whoever unlocks AGI will unlock even more growth 3. Even when you unlock AGI, you'll want to throw gobs of money at it to improve itself and all sorts of things.

> If Fable comes out and demands 50x the price of DeepSeek in order for Anthropic to make a profit on it, how much more productive would I be compared to my personal experience + DeepSeek? 3x? 50x?

You're pricing it wrong and looking at it wrong. First, the per token price doesn't consider that a smarter model can end up using fewer tokens overall to achieve a result. Secondly, if the difference is between failing to accomplish the task and accomplishing the task, suddenly that 50x can seem like a bargain.

> Is it cost effective for a business to hire someone without SWE experience + Fable verses hiring someone with SWE experience and DeepSeek? When does R&D hit diminishing returns?

At this time, someone without SWE experience + <name AI model> vs someone good with SWE experience and <name another AI model> is a no-brainer. The AI model is an accelerant but the "no SWE experience" will be accelerated into a wall. Now maybe that doesn't matter for prototyping and certain other things, but anything in production the lack of experience will hurt them with things they won't even know about or even know how to look for it (e.g. slow, insecure, etc).

> You're pricing it wrong and looking at it wrong. First, the per token price doesn't consider that a smarter model can end up using fewer tokens overall to achieve a result. Secondly, if the difference is between failing to accomplish the task and accomplishing the task, suddenly that 50x can seem like a bargain.

Perhaps, yeah. Napkin math; if we go off of the assertion that DeepSeek makes me twice as productive for an extra $24 per year.

The same number of tokens with Fable would be around 50k/y. Say Fable uses half as many, that's still 25k

The question is then; How does Fable make an engineer more productive than DeepSeek?

- Prompt adherence?

- Inference speed?

- Can engineers delegate more work to unsupervised autonomous workflows?

Are unsupervised workflows suitable for production applications or does the review overhead erode the productivity benefits?

Perhaps a company could hire a mid level engineer + Fable to achieve senior engineer levels of productivity for the same price as a senior + deepseek.

> Even when you unlock AGI

Massive assumption there.

For clarity, inference is typically a COGS and therefore hits Gross Margin vs model training which would typically be in OpEx (where R&D lives) and would hit operating margin.

Even if you discount superhuman AI (which I would emphasize that frontier researchers do not discount and expect to see soon) think it’s still hard to have enough confidence that the ground is solid. Someone in 2024 trying to go down this route would have invested a lot of now-pointless effort into prompt engineering.

> Unless we are genuinely pushing to find AGI, at which point nothing matters, LLMs in their current form don't replace knowledge workers but are an effective force multiplier. How good is enough?

There's a non-negligible percentage of the industry who have a pseudo-religious belief in AGI, so I wouldn't be surprised if that was, in fact, the goal.

Who knows, maybe they'll stop once the money dries up.

If a model comes and makes developer + Deepseek even a little more productive, from employers perspective, it'd still make sense to pay a lot of money for that.

Deepseek shines for personal usage because it's possible to use it however you want and whenever you want with no session/weekly limits stress because you use the API and it's priced very reasonably.

> Unless we are genuinely pushing to find AGI, at which point nothing matters

I think the third coming out Jesus Christ in closer than AGI. Seriously, I dread how much of Silicon Valley is wrapped in this narrative of AGI and Singularity.

How can all these "rationalists" fail to see that this is what religion looks like: Faith and promises of heaven and hell.

These "rationalists" understand that beliefs should be evaluated on whether they match what we observe, rather than preconceptions about which buckets certain ideas fall into. If the Southern Baptist Convention had announced a theological breakthrough in 2022 that lets them map out the precise calendar dates of the events in Revelations, and using this map they made a series of specific predictions that ended up coming true, it would be rational to start studying End Times Christianity in more detail and irrational to say that it's religious so you're not going to worry too much about it.

That's interesting on Deepseek. But I think as long as the models are still making noticeable gains with each iteration it's hard to say "good enough."

A thousand 90 IQ cannot do what a 145 IQ can do.

Similarly, some bosses might believe that they can hire 100 cheap, unmotivated SWEs to replace Linus Torvalds or Fabrice Bellard and achieve something slightly worse. But in certain areas, it doesn't work like that.

i agree with this

on the other hand, the sad reality is many swe are working on dumb crud apps and the code-quality is already very low ime, and the jury is still out if ai tools can long term replace those; but what i have seen first-hand isnt super promising do far...