Some suck. Google's Tensor is an example of one that doesn't, that some consider superior to Nvidia's offering. There are likely others.
Some, but how much? Once you have a mass produced ASIC with a model that's "good enough" much (but obviously not all) of the demand for the cutting edge could evaporate quite rapidly I think.
I'm unsure about the energy efficiency of Tenstorrent cards, but they are at the moment much more cost effective than Nvidia. But software support is far behind, I suspect if it wasn't the cost would be more equal.
No. Per Jeff dean.
Within Google itself the TPUs are definitely preferred for AI workloads, you just can't always get them.
I mean, Google tell me it's TSMC's 3 and 2 mm process - exactly what you'd expect. But you can Google as well as I can.
You can see the process node table here.
The newer chips are probably 3nm and they are probably developing the next chip in 2nm.
The writing was on the wall for TSMC ever since the 5nm node; Apple proved that anyone with enough liquid cash could block competitors from fab access.
TPUs are significantly less flexible and I would guess at an individual level not as efficient or fast as Nvidia. But they're designed specifically to be bundled together in large arrays with similarly large data pipes to storage and interconnects to cache, ram and other compute. I would guess Google didn't bother trying to sell them until recently because you needed to be deploying them at large scale for it to be even worth also developing the software for.
What we really need is innovation in super fast (bandwidth and latency) and reliable interconnects. I suspect the days of slotted in GPU and RAM will be over soon and we'll see a move everywhere to pin based sockets (like we've always had for CPUs). Stuff like SXM and LPCAMM
They're not buying up "all the supply to build an inferior product". They're buying it up build a different, high-demand product.
Google's used this strategy a few times before, and it frequently doesn't work out. They kept MapReduce and GFS proprietary so they could build better big-data services than everyone else, but Doug Cutting from Yahoo clean-roomed it, and then eventually you had interviewees going "MapReduce? That's like Hadoop, right?" They had a long string of internal ML frameworks well before the rest of the industry (I remember in the '00s and early '10s Google was by far the leader in large-scale machine-learning) but had to open-source TensorFlow in the face of external competition, and then lost in the marketplace to PyTorch because TensorFlow was filled with internal WTFs and footguns needed to support specific Google projects. They had a big lead in datacenter technology over everyone else in the industry in the early 2010s, but kept it internal and proprietary until they were forced to sell it as a service by the huge growth of AWS and Azure, then had to play catch-up as a distant 3rd because they lacked the customer relationships and developer mindshare.
Both are fabless customers of TSMC, which happens to be the choke point.
Unlike gp, I do not believe its a market failures as much as the cutting-edge process node bieng a natural monopoly. Once TSMC processes got really good, TSMC started capturing most of revenue dollars and reinvesting them, which made TSMC products even better - a situation that makes it impossible for Samsung or Texas Instruments to catch up. Intel rode on a similar wave for decades
https://www.insidedeeptech.com/tsmc-cowos-packaging-full-gui...
I think even if you have the capital to acquire something from ASML… by the time it’s setup and tuned correctly TSMC will likely have enough capacity to eat your lunch. If it takes 2-3 years to get a fab online TSMC will likely have a new node and their old ones will be available.
TSMC has really proven over the past 5-10 years that the fab is its own beast and requires its own immense level of expertise and investment. When you have the worlds largest chip companies shoveling all their manufacturing dollars through one single org, that seemingly does the right thing, it’s going to be pretty much impossible to beat them.
Literally TI even uses TSMC for the digital only stuff they do. The investment for these cutting edge nodes (or even, not so cutting edge. Like say, 10nm) is absurd vs the payoff.
The main thing is that chip manufacturing is the most highly capitalized industry in the world. Each manufacturing step has gargantuan costs and so no company wants to manage (and pay for) all the steps - so TSMC buys chip manufacturing equipment from ASML and sets up a system where corporations buy capacity. It's essentially a very, very, very expensive print shop. The print shop doesn't make its own printers nor it print a newspaper.
And naturally, with very, very objects like leading-edge chips, "inferior" or "superior" is a ridiculous simplification. Nvidia chips are incredibly powerful and useful. Nvidia has been more committed than any other company to general purpose GPU programming.
But Nvidia also is doing it's best to sell at high, monopoly prices. That's why they buy a lot of capacity. The thing about the situation is these companies did spend a lot of money reaching this monopoly situation so from their viewpoint, this monopoly pricing might be fair. But I personally wouldn't care and hope some regulation stops it. But that kind of regulation would still have to get these companies significant profit on these investments, so they'd keep making them.
If I come to a coffee supplier and say I can pay 50$ for a sack of coffee but I don't know how many I can buy yet as I am building up my coffee shops around the country and then Starbucks comes in and says we are willing to pay 55$ and pay ALL your capacity guaranteed, today, just sign here.
Where do you think most coffee sellers would sign?
You've got it all mixed up.
Google designs the TPU in partnership with Broadcom. Many companies will write the Verilog RTL and then send this to another company like Broadcom to do the physical design and turn the Verilog along with lots of other IP blocks like serdes into GDS mask data.
Broadcom does not have any fabs. They send that GDS mask data to TSMC to manufacture the chip just like Nvidia, Apple, and tons of other companies.
I'm a physical design engineer if you have any questions.
Inference has already been commoditised.
https://d1.awsstatic.com/onedam/marketing-channels/website/a...
But the next gen of Tenstorrent HW looks great on paper - problem is, I could have written that before the current gen was released.
All of the allegedly competing HW looks like that and has looked like that for years, either a lack of availability, capability, or both.
Already these are run on different hardware, at least as I understand, and the computational needs could further drive differentiation.
https://www.forbes.com/sites/jonmarkman/2026/08/09/amd-buys-...
It is available to Oracle.
https://www.oracle.com/news/announcement/ai-world-oracle-bri...
Why? Don't they want to add another $3-5T of market cap?
Do we actually know they're more efficient per watt or for the power used and heat produced ? Or are they just easier to scale (and manufacturer) en masse ?
There's also the CUDA moat which is hard to get around
Nvidia's moat for LLMs isn't Cuda, it's installed capacity, arrangements with frontier labs, TSMC bandwith, and just the fact that it currently has no real competition.
This Huang scandal has so many dimensions that are clearly more impactful on the world outside than to NVIDIA itself -- the blast radius seems to extend to the US cabinet -- that it begs questions if Jensen Huang wasn't id'd as the essential piece to remove to affect a comprehensive restructing in the segment.
So if deals fall apart because Jensen and his handshake have left the room, that may unclog many memory and processing supply lines that are currently spoken for by deals with Jensen. That may have been a goal from the start.
But once you do that, translating CUDA into one of many available vectorized backends is trivial.
We have a bunch of custom kernels for geometric operations (like merging polygons) and they were rather difficult to write a while ago. AI rewrote them in HIP during my lunch break.
those Asics have a fixed algorithm
Cerebras already does that an need to make a new chip for each new model
that's why you couldn't make an ASIC for Ethereum
I'm kinda bewildered, it's bizarre seeing people cheer on quickly-obsolete accelerators instead of demanding their OEM to adopt GPGPU. CUDA will be relevant for much longer than any ASIC can ever be, even (or especially) if the transformer gets replaced overnight with a better architecture.
Ethereum's work function was designed to be memory bandwidth limited, which is something gous are already quite well optimised for. An asic would be faster but not by enough to be worth the investment, at least while gpu margins were less crazy.
The main thing is the actual number crunching and memory interfaces for LLMs are not crazy hard to implement, and the potential scale is huge, so NVIDIA's moat is not likely to be big enough to make their current margins sustainable in the longer run.
My background is ASIC design and more recently 3rd-party IP development for integration into ASICs. In our last design I made heavy use of resource sharing (RAM and holding regs for intermediary values, counters, LUTs, complex multipliers, adders, etc, even state machines).
That got the wheels turning. I believe such an approach can be generalized to support a wide range of computation and thus algorithm development. Even state machines can be configurable if designed that way. All controllable by programmable registers.
Just don't look for AI to take on such a task. Or go ahead and try, it already fails miserably when it comes to RTL design (verilog, vhdl). LOL
one in the "Cerebras" direction of ever-larger scale general purpose HPC silicon running inference workloads. Cerebras may have skipped ahead to the last act of that scaling story, notwithstanding any further wafer-diameter enlargements past 300mm. Whether the ending is happy or not is still unclear. Lot of iterative room to grow into it further and they have a formidable patent moat waiting for conventional die-sized AI semi vendors when they inevitably run up against limitations of single-die designs.
the other is in the "Taalas" direction of freezing a specific model into something like high speed ROM / FPGA and getting orders of magnitude inference speed improvements - for that specific model. Like Cerebras, Taalas may also have jumped directly to the architectural telos of that philosophy, with only iterative improvements in cost per work unit left to grind, though like CBRS quite a lot of grinding left to do especially in getting cost per custom tapeout down to near-disposable (hopefully, recyclable) levels.
Yeah I was thinking this will probably be the case. We may eventually settle into a slower release schedule as well, where a new model only comes every year or two.
Bitcoin was lucky for Nvida, the AI was lucky. Third times the charm?
On other hand, luck favors the prepared. Beyond luck, they did make their chips accessible, easier to program than other GPU's. So bitcoin miners and then AI could utilize them.
There are still a lot of people that want to run regular liner algebra.
Nvidia cards were the worst hashrate/purchase price of any cards. AMD were the lucky ones there if you consider anyone lucky in this situation. Consumers definitely weren't lucky because the GPU market hasn't recovered since the Etherium boom.
I imagine the 1070/1070ti 1080/1080ti were decent at mining, but a lot more expensive to obtain for these gpu mining rigs with 6-10 gpus in them.
I wonder if cooling such a beast, with very non-uniform hot spot distribution, would be too complicated to bother with though.
That function block was worth the cost of accusation.
But Bitcoin mining moved GPUs to ASICs many many many years ago and Ethereum switched from proof-on-work using GPUs to proof-of-stake: GPUs are pointless now for Ethereum. And Ethereum was the big one for GPU mining. All the others cryptocurrencies are too tiny to register.
Ethereum switching from proof-of-work to proof-of-stake has been the death blow to GPU mining at scale. I'm not saying some have not been profitable mining other less important coins: I'm saying the market for GPU mining shrank to next to nothing.
Training? I'm not so sure. You need something flexible so that it can be trained. Burning the weights into the board is the opposite of that.
At these dimensions, you are better off producing general-purpose inference ICs that can run any model, instead of baking in a single fixed LLM.
The thing is, it's only NVIDIA really that has a proven product in high volumes pumping through TSMC with the existing software stack that the community has already proven. That's a huge difference. Everything else is riskier if you're building out in volume. For now.
Relevent article from september 26. To me it’s obvious gpu are a « first response » solution. But I think nvidia can sell a lot before AI R&D slow down (because of the demand or the tech, infra, energy… or whatever reason it will be)
They still buy Nvidia and x86 chips but this is what the WSJ article from last week was saying. The hyperscalers are moving away and Nvidia is committed to TSMC wafer orders.
Microsoft
https://blogs.microsoft.com/blog/2026/01/26/maia-200-the-ai-...
https://techcommunity.microsoft.com/blog/azureinfrastructure...
https://cloud.google.com/products/axion
Amazon
https://en.wikipedia.org/wiki/AWS_Graviton