Genetically and physically yes, you would be way closer to a H. Sapien from the other side of the world than you would be to a Neanderthal who lived a cave over.

Perhaps culturally a geographically close Neanderthal would be easier to relate with than a more distant modern human, but that's pure speculation.