U3 is a fanciful number, both because people are using disability insurance as a way to collect benefits indefinitely and because once you give up looking for a job you don't count as unemployed. The US has the lowest labor force participation rate in decades.
In terms of European countries there was an article about a week ago regarding Americans going to Germany for free college. The Germans were hoping they'd stay.
I think people worry about the quality of immigrants. We want more young Sergei Brins and fewer middle aged guys with a sixth grade education.
One could argue that saying that you're against illegal immigration and making legal immigration very difficult is the same as saying that you don't really want immigration.
Care for the elderly is still an intensive task. Maybe yes, in 30 years it will be done by (Japanese) robots but we're not close yet.
You have the economic burden too: how much more productive each working individual has to be if it has to offset an increased ration of non working individuals?
Lastly, I don't think a country is sustainable with a dwindling population unless it can be physically and culturally isolated and has an important deterrent against military action