That overlap is more than large, it's actually the most important factor in the calculation. You only need to go back about 6 generations (200 years), and there is practically no interchange between geographically separated groups, which means that I probably don't have any genes from some random asian living 1000 years ago (although we both almost certainly share genes that we inherited from some far more ancient common ancestor).