At least for gaming, GPU performance per dollar has gotten a lot better in the last decade. It hasn't gotten much better in the past couple of years specifically, but I assume a lot of that is due to the increased demand for AI use driving up the price for consumers.
Why wouldn't Moore's Law continue?
With GPUs and CPUs. You need to replace entire thing. And now they are the most expensive part of the system.
Other option is doing more with same computing power, but we have utterly failed with that in general...
A few thoughts:
1. The comparison to previous tech bubbles is apt - we're seeing massive capex without clear paths to profitability for many use cases.
2. The "build it and they will come" mentality might work for foundational models, but the application layer needs more concrete business cases.
3. Enterprise adoption is happening, but at a much slower pace than the investment would suggest. Most companies are still in pilot phases.
4. The real value might come from productivity gains rather than direct revenue - harder to measure but potentially more impactful long-term.
What's your take on which AI applications will actually generate enough value to justify the current spending levels?
We are not seeing that (currently) with GPUs. Perf/watt has basically completely stalled out recently while tokens per user has easily increased in many use cases has went up 100x+ (take Claude code usage vs normal chat usage). It's very very unlikely we will get breakthroughs in compute efficiency in the same way we did in the late 90s/2000s for fiber optic capacity.
Secondly, I'm not convinced the capex has increased that much. From some brief research the major tech firms (hyperscalers + meta) were spending something like $10-15bn a month in capex in 2019. Now if we assume that spend has all been rebadged AI, and adjust for inflation it's a big ramp but not quite as big as it seems, especially when you consider construction inflation has been horrendous virtually everywhere post covid.
What I really think is going on is some sort of prisoners dilemma with capex. If you don't build then you are at serious risk of shortages assuming demand does continue in even the short and medium term. This then potentially means you start churning major non AI workloads along with the AI work from eg AWS. So everyone is booking up all the capacity they can get, and let's keep in mind a small fraction of these giant trillion dollar numbers being thrown around from especially OpenAI are actually hard commitments.
To be honest if it wasn't for Claude code I would be extremely skeptical of the demand story but given I now get through millions of tokens a day, if even a small percentage of knowledge workers globally adopt similar tooling it's sort of a given we are in for a very large shortage of compute. I'm sure there will be various market corrections along the way, but I do think we are going to require a shedload more data centres.