mbesto

> you offered up a case study, with incomplete data, as possible evidence that AI is generating these financial payoffs.

You're confusing the original article and the parent comment I replied to. I think the Apollo article is way off, BUT I have very intimate knowledge with very concrete numbers of a use case where AI has a very clear and definable ROI.

> Could. But again, are they?

YES. That's what I'm telling you. They just implemented the system. The contractors spend less time on each project which means they have more time to do another project. How is that so hard to understand?

> You're extrapolating a 3x increase in throughput from the optimization of a single process that exists in a whole business system.

Are you deliberately being obtuse? The company very clearly drives ~$5k additional profit per project. They do hundreds of similar projects.

> Also, do we think the S&P 493 is so incompetent that they'd miss these optimizations? Or is it more likely that this firm you're referencing is a small inefficient player, and the S&P 493 already realized these efficiencies decades prior without AI?

I never made any of these claims, why are you saying I did?

> They have infinite money to burn so they aren't going to optimize.

They also kind of ("kind of" being the important clarifier here) can't optimize. See this is the trend with agentifying companies and the achilles heel for the frontier LLMs. We're starting to see diminishing returns on their capabilities, which means the rational enterprise is going to simply use the cheapest most efficient model for the task at hand. In turn we can just use an agent broker to decide which model is best and cost the least. Anthropic (today) has no incentive to do that other than to just use their own, best model for its own work.

BYOK (bring your own key) is the term the parent meant to say.

> I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.

If the outcome is the same then the fees shouldn't matter to the client in the long run. Said differently, if the client who pays the management consulting fee gets the same return out of the service, why do they care?

Now, what may end up happening is that some other management consultant vendor in the market comes in an says "ya know what we're willing to do the same work for less" (which means their margin goes down). This could indeed drag their pricing down.

> Interesting times on vendor contract negotiations these days.

For sure.

The consultants are contractors. Even if they were FTEs, it would mean the consultants could do ~3x the work.

You seem hellbent on disproving me. Why?

The firm charges a fixed fee to the client. The client doesn't see anything different from what they got before, the firm increased their gross margin overnight.

I know a very specific use case in management consulting where hours spent by a consultant for a repetitive task (mainly report writing and source data gathering) went from on average of 40 hours to 12 hours per project. It costs the firm roughly ~$150 in token spend to save those 28 hours. They used to be roughly $200/hr (so $5k~$6k).

That's about as hard of data as you can get.

If models got smarter and an average prompt response used let's say 5k tokens instead of 10k tokens, would that change your prediction?

Point is, the number of tokens is a silly metric to proxy these companies' growth.

Isn't prompt injection just one facet of the trifecta? Like, I just don't know how it's remotely possible to have a non-deterministic block box ever deterministically know whether your credit card has been compromised or not. Even if you were hacked, how would you even know?

Am I missing something, how is there any sort value in these assistants acting as agents on your behalf when they are basically a black box? Like if it can't use my drivers license or my phone number or my credit card, it can't do 75% of things people claim it can do without being massive security issues. What am I missing?

For the record I use OpenClaw but it has its own services (gmail address, phone number, etc.) and I don't give it any of mine, which inherently limits its capabilities.

tptacek oddly silent here...

Ding ding

> Why would i bother? I pay $100/m for the frontier model, understanding it's marginally better than much cheaper alternatives. I get so much value from the product it's not worth shopping around

Wait until you are forced onto an enterprise plan...

They're already useful. People are shipping real products with them.

Also, it's entirely plausible that an previously open model (that then became closed) will be much easier to distill than having a bunch of closed models.

I agree it's hard to predict.

It's very possible given the current environment that LLMs go the way of airlines. Low margin businesses that are essential to world economics with heavy price competition.

If thats the case, the parent's critique was oddly stated and confusing. The phrase "to do it" implies the "LLM puffery" was required to accomplish the task, which clearly providing LLM generated praise isn't required in either setting to create a world in minecraft.

> Collecting incomplete or incorrect data and adding a hunded disclaimers at the end of the report is standard LLM behavior. Actually valuable analysis of this sort needs a lot more human effort.

Or it's classic modern journalism that gets clicks. LLMs are just mimic'n human behavior.

> No previous tech shifts are comparable.

Oh please, do tell.

You people realize businesses used to have rooms full of humans who clacked away at typewriters right?

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg...

This is pure speculation. This is like saying blockbuster films no longer have a camera operators rigging pro level cameras because sony came out with the alpha series. I suspect that the number of new people NOT using IDEs (e.g. using Claude Code, Codex, etc.) is growing rapidly but it may not necessarily mean that less people are using IDEs.

> Will it? Are you willing to test this?

Yes, this literally happens all of the time today. Proof point - software development organizations hire to fill operating budget voids, not on what work needs to be done.*

*Source - I've seen 900+ software team's P&Ls and interviewed their CTOs.

This is a pretty pathetic analysis especially for an analysis from a firm like Apollo. A few notes:

1. Profit margins from the mag 7 are on an upward trend starting in 2020. This is pretty consistent with the insane topline growth had by covid and the most recent layoffs occurring over the last 6 years. To unilaterally attribute this to AI implementation is lazy.

2. Most non-tech firms are only implementing AI just now because we finally have better enterprise tooling required. We won't see the effects for another 12-24 months at least.

3. As a former management consultant, this might be the worst use of a stacked bar chart I've ever seen: https://www.apollo.com/content/dam/apolloaem/images/daily-sp...

4. I don't know how you actually isolate AI productivity gains on a macro level. It seems pretty much impossible.

This whole thing is laughable.

$200...PER YEAR.

For coding or what?

For general purpose LLM, you can't beat Gemini AI Pro. It's $200/year and you can add your whole household.

> The advisors who suggested it are long gone from SoftBank

You mean Masayoshi Son, the founder and CEO of SoftBank? lol wut.

https://economictimes.indiatimes.com/news/international/us/s...

> Softbank Vision Fund is one of the most heavily ridiculed and mocked international VC funds anywhere in the world.

Y'all need to stop reading TechCrunch headlines.

> When you can throw billions around for investments then on balance you will make a lot of money because you can take a lot of risk.

How does this explain a $20M bet on Alibaba that netted $60B?

> if you look at the ratio of good bets to bad bets then SoftBank is terrible.

So does a16z, Sequioa, etc...

This is venture investing 101 - they have power law returns. Don't let your survivorship bias cloud your vision.

Even easier with https://colima.run/ if you're on MacOS

This is objectively not true. Alibaba, DoorDash, ARM, Slack, etc. were all SoftBank deals. Alibaba alone is considered one of the highest ROI's in the history of investing.

Does SoftBank have some absolute blunders on large bets? Totally. Do they make some super weird billion dollar bets? Absolutely. Can you scrutinize them for making some terribly and laughably large bets go to complete zero returns? Sure. But this would be like saying Michael Jordan had a "bad track record" for scoring points because he had games playing for the Wizards where he had like 5 or 10 pts in a game.

On paper, this "check" for this is that people this corrupt don't get voted in. But alas.

> The rate at which we're churning out AI-generated code is going to make this worse, too, because AI-generated code is kind of notoriously terrible at security.

This seems incongruent with what we're witnessing. Right now we're witnessing vulnerabilities that are 6 years old be exposed. That code wasn't generated by AI slop.

Security has always been about attacker/defender symmetry. So if the attacker has an LLM so does the defender. Reality is, the tax is getting higher as security tooling extracts more perceived value.

> Our American obsession with jail is nuts.

Sure but also there has to be some sort of deterrent and an executive who's worth is $100M+ who can't touch their $100M because their in jail is far more of a deterrent than META missing its earnings for a quarter because they owe the government a one-time slap on the wrist.

I'm all ears on what a different penalty could be that ultimately stops this behavior.

PS - META's stock is currently up 2% in after hours trading, so