My read is that investing into AI is the trendy thing to do in 2024. No CEO will have to justify pouring money into AI; they would probably be seen as negligent if they don’t. It’s just the zero-risk strategy at this point in time. That’s not to say that there is no value to be realized, it’s more a question of how much value and at what price. If in the next 12-24 months we find out that AGI, robotics and the end of labor is not as imminent as some have hoped and that perhaps not every car dealership needs an AI chat assistant, and the first companies announce reductions in their AI spending, the dominos might begin to fall. Or not, let’s see.

And yet, once the money dries up after chasing another pointless hype, the "visionary" CEO will still keep their job - while starting yet another round of layoffs.

If anything, the post you are responding to explains why CEOs have to think this way, they have no real choice. No one really knows where AI is going, yet staying on the sidelines is not an option.

The job of a CEO is to make sure the company is viable and liquid longterm, communicating their thinking.

"Society made me do it" is not a strategy.

In my opinion still to be seen where LLMs are going in general. They are cool sure, but I don't see its value measured in trillions in 2024 but my prediction and investment aptitudes are disastrous.

Replacing search is enough to warrant trillion dollar valuation for LLMs, question is just if they can get that good.

Note that you need to replace search on the lucrative queries, stuff like you want to buy a product or want to know about nearby open stores, those are the queries Google focuses the most on and will be the hardest to replace them on.

> replacing search

But since Google is adding LLMs to search, an LM competitor might not win still.

> lucrative queries

Also the hardest for an LM.

I predict this won’t end well for anyone. Google will destroy a bunch of their own value preemptively self-disrupting and burning a ton of cash only for everyone to discover that it’s not lucrative to use an AI to tell people the distance to the moon. Viagra and insurance queries will stay valuable and be least useful for a LM to disrupt. Google will maintain their market share, at the expense of more cost per query, so they’ll purge more of the loss-leaders across their consumer offering. We’ll all discover that an LLM isn’t great for search and better integrated into Gmail and similar.

> Replacing search is enough to warrant trillion dollar valuation for LLMs, question is just if they can get that good.

I would add that it needs to be more than just good, it also needs to be cheaper for mass adoption.

Spoiler: classic search is at least cheaper (old post, but relevant: https://www.semianalysis.com/p/the-inference-cost-of-search-...). The thing is it cannot get that much cheap without disrupting earning expectations from hyperscalers.

they're basically already that good in my opinion, except for a few subsets of realtime+hyperlocalized information. i feel confident that is addressable

Even if current AI progress caps out at GPT-4 level capability, we are only scratching the surface of how much application integration there is going to be. Until AMD or someone catches up, inference could easily 10x demand from where it probably currently is.

I think you're right that demand for inference is likely to grow by orders of magnitude in the medium term, but I don't know that Nvidia's lead over their competitors is likely to remain as strong as it is today. That is, they'll probably hold their dominant position, especially for training, but I'd expect their lead on inference to shrink by a lot. Companies pretty much understand how to make inference engines. Expect AMD, Intel, Qualcomm, Apple and others to come out with products geared at fast and efficient inference. Not to mention Grok and other dedicated chips.

I used to think this. I still mostly think this (and have frankly lost a lot of money in opportunity cost because I thought it), but I've been astonished at how long AMD is taking to pivot, such that there are startup competitors like Groq that might honestly beat them out.

And let's not forget, this is a comparison between NVDA and AAPL.

It's not just a matter of NVDA's upside.

AAPL may have peaked, and, in light of the EU breaking up it's walled garden, may be on the way down.

Well, then let's hope EU puts its eyes on NVDA and CUDA, another walled garden.

CUDA is a technology investment that gives Nvidia a moat, but there is nothing exclusionary about it, Nvidia invested in tooling years ago ... it will just take time for others to catch up.

NVDA banned CUDA translations layers, that's pretty exclusionary because it denies interoperability. There was already a discussion about it here: https://news.ycombinator.com/item?id=39592689

Well, they tried to legally prohibit it, but as people in that thread pointed out, it's legally irrelevant in most developed countries ... so another hardware maker can fund a (legally) independent team in say France (most favorable reverse engineering laws) to break it / copy it / do whatever they like ... and then open source the output for the rest of the world.

It's even dubiously legal in the US ... but there it would be a war of legal fund attrition and Nvidia has deep pockets.

Still - break it in France, give it away ... US laws go poof.

It's really just the ongoing development of the software ecosystem that gives them their moat ... but it's a good technical moat because they steer future dev and others play catch up.

Actually good points. I just hope other players catch up.

Microsoft just announced that they are getting better price/performance ratios with AMD for inference. It's happening already. Most of GPU thirst for training at the moment but eventually most of workload will be inference. And that's depending on a big if, if users are willing to pay for LLM generated content as much as we're hyped about it right now.

Is that “ballpark”? It’s a 3x multiple.

I do agree that growth will cap out. A lot of the massive growth is due to the previous fiscal year being effectively a crash. It’s impressive that they’ve recovered but the same quarter next year will not have the benefit of a bad quarter to compare against.

> I can't see these earnings being sustainable for nvidia over the long run.

We've barely scratched video and volumetric video training, even if text stuff hits a data wall I think demand will stay high for quite some time due to that.

The move to on-device inference actually takes more training, you need to train it longer to make it still perform well at edge parameter counts (with lots more training, Llama 3 8B is competitive with 70B). Nvidia is already more specialized towards training and has lots of competitors in inference.

Training tends to have steps that don't just involve matmul; Jim Keller said Tesla Dojo ended up having problems from being too rigid, where GPUs have been able to adapt to new training regimes better, though Google's TPUs have seemed to not fall into that as much, maybe just iterated much faster?

*llama 2 70B

Unless AI demand increases? Which seems likely, or at least possible.

Or decrease. It looks like there is 200B industry is already sucking funds, but no end-user app with strong revenue stream is on the market yet.

And then consider what AI content spam might do to ad revenue, which many AI-adjacent companies depend on.

Profits even closer: 15b vs 24b

Nvidia has higher margins so it first need that much growth to become the world’s most profitable company.