It's a very focused and impressive effort. Training from SRAM can be one-two orders of magnitude faster than training from DRAM.

Modern GPUs, even with some process advantage (and no shipments her, H100) strive to be general purpose processors too much, and that caps their peak deep learning performance.

Every serious player will have to make or buy their own TPU.