While that's true for simple conductive cables that's not the case for semi-conductors.
https://en.wikipedia.org/wiki/Saturation_velocity
Especially since there are some semi-conductors which actually require you to move electrons, like e.g. flash memory ;)
TL;DR: "The speed at which energy or signals travel down a cable is actually the speed of the electromagnetic wave, not the movement of electrons. Electromagnetic wave propagation is fast and depends on the dielectric constant of the material. In a vacuum the wave travels at the speed of light and almost that fast in air."