The speed lost due to distance would be infinitesimal, but is that the only speed loss there would be? I'd expect that parasitic inductance and parasitic capacitance would be higher in a socketed system which would impose speed limits.