I'm kinda bewildered, it's bizarre seeing people cheer on quickly-obsolete accelerators instead of demanding their OEM to adopt GPGPU. CUDA will be relevant for much longer than any ASIC can ever be, even (or especially) if the transformer gets replaced overnight with a better architecture.
Ethereum's work function was designed to be memory bandwidth limited, which is something gous are already quite well optimised for. An asic would be faster but not by enough to be worth the investment, at least while gpu margins were less crazy.
The main thing is the actual number crunching and memory interfaces for LLMs are not crazy hard to implement, and the potential scale is huge, so NVIDIA's moat is not likely to be big enough to make their current margins sustainable in the longer run.
My background is ASIC design and more recently 3rd-party IP development for integration into ASICs. In our last design I made heavy use of resource sharing (RAM and holding regs for intermediary values, counters, LUTs, complex multipliers, adders, etc, even state machines).
That got the wheels turning. I believe such an approach can be generalized to support a wide range of computation and thus algorithm development. Even state machines can be configurable if designed that way. All controllable by programmable registers.
Just don't look for AI to take on such a task. Or go ahead and try, it already fails miserably when it comes to RTL design (verilog, vhdl). LOL
those Asics have a fixed algorithm
Cerebras already does that an need to make a new chip for each new model
that's why you couldn't make an ASIC for Ethereum