You're not wrong, but the struggling countries may not have the infrastructure to fully take advantage of the skills and talents of the 'best and brightest'. By going to Canada (or wherever) they could be leveraged more effectively.

A lot of technology is making things easier in struggling countries, and if that technology can be made cheaper and commoditized it can improve more lives at more points of the income spectrum. The best and brightest may be helping to eliminate their own positions by making certain jobs easier to do in more places on the planet.

Right, on one hand if you wanted to maximize everyone's contribution, a 'top performer' moving to a rich country and being more productive there is ultimately a good thing. And like you said, if their work leads towards equalization between the two countries its even better.

On the other hand if the output of their productivity isn't flowing back to their home country, Canada/USA/UK/wherever are reaping the rewards tax/tariff-free and the increase in productivity isn't being redistributed.

You wouldn't let a foreign power extract oil or gold for "free" from your country, would you? I don't want to argue against personal liberties but at the same time it could be argued that those rights are being exploited by rich countries to the detriment of poorer countries.

Then there's the whole TFW (temporary foreign worker) program that looks a whole lot like serfdom where landowners in Canada buy labour at below market costs without even extending the notion of citizenship to the workers...