you are most likely correct! I read ´Training took 1.82 minutes with 256 fourth-gen TPUs’ to mean Using all 256 TPU it took 1.82 wall clock time to do the training not 466 minutes.

This mean each TPU contributed to (1/256) assuming it scale linearly.

Even if it doesn’t the 256 TPU ran in parallel not sequentially right ?