We often think of 'Tea' as a very English thing (and it is in many ways now), but of course it has it's roots in China/Asia i.e. global trade.
The Japanese borrowed ideas from elsewhere for quite some time (as has everyone of course), but they would view 'building Western warships' as something very different from 'importing millions of Westerners'.
Also, one could argue that Japan, for a variety of reasons, was far more isolated than other places.
You're probably going to have a hard time finding a job in Japan that isn't "English Teacher" if you can't speak Japanese effectively - which is an example of the parent culture exacting a price in the form of partial assimilation for the opportunity of cultural exchange.
Plus, most immigrants to Japan are probably going to be from China, Korea or other Asian countries, simply because of proximity. Immigration doesn't necessarily have to mean millions of Westerners.
All cultures change over time, even the ones that some people, for some reason, hold up as icons of homogeneous purity like Japan. A culture which isn't an expression of the people who currently live in a certain place and time is dead. That immigration would inevitably alter Japanese culture is a sign of health and evolutionary progress, not destruction.
I mean, the Ainu might even consider it karma, since they were already there before the Yamato showed up with their weird foreign ideas.
[0]https://news.ycombinator.com/item?id=13693800
[1]http://www.japantimes.co.jp/news/2016/10/05/national/history...