Yes, but remember the "bottleneck" from 70,000 years ago, when the human race was nearly wiped out. At that time, because of a nuclear winter brought on by a volcanic supereruption, there may have been fewer than 10,000 humans in the entire world.
http://en.wikipedia.org/wiki/Toba_catastrophe_theory
So, based on that, we're descended from fewer people in the past than most people realize. Another way to say it is that most of our ancestors were cousins.
That overlap is more than large, it's actually the most important factor in the calculation. You only need to go back about 6 generations (200 years), and there is practically no interchange between geographically separated groups, which means that I probably don't have any genes from some random asian living 1000 years ago (although we both almost certainly share genes that we inherited from some far more ancient common ancestor).
If a generation is about 30 years then you will have close to 33 ancestors lines in your family tree over those 1000 years. That makes 2^33 ancestors or 10 billion, that's impressive (well there is probably a large overlap)