Eventually you can't stuff enough computing in a small area (power density). Therefore you have to connect multiple CPUs spread out in space. The limit for many supercomputers is about how long it takes light or electrical signals to travel about 20 meters. Latency to first result is only part of the measurement that matters.