I know that's impossible for many reasons, but which body would you wake up in if somebody would make an exact subatomic-perfect duplicate of your body. Wouldn't the copy become just a twin having identical memories and believing he's you? Wouldn't you still only see the world from the point where you've been rather than where your copy has emerged?
Maybe you'd be the same in a fraction of an instance in time?
Maybe you'd never be the same because the process of duplication is an experience within itself enough to develop a different self?
Who knows :D
Some back it by the change of personality after major trauma.
Moving a mind to another body is most definitely quite a trauma (maybe the ultimate trauma?), which begs to ask the question:
What's the point of moving your mind to another body, if you'll cease to exist as you are? If you lose, like you said, the "sense of you".
That's why a lot of people think a human brain would never be able to be uploaded to a computer, and if you could do it, you'd not be yourself - you'd be something else.
You are the sum of your experiences, a build up of scars, traumas, and so on.
So according to this theory, you'd only wake up on someones else body if that person had the same sum of experiences you had, yet that's impossible because only you are you - not even twins would experience this. Maybe that's why we value this experience so much: it's so unique and so intimate to ourselves, that we hold on to it dearly.
You then have hints of collective (un)consciousness. Jung explored this concept with archetypes.