They're more than happy to charge thousands to qualified Eastern/Southern European to get them to come here and pocket high salaries and relatively cheap cost of living / taxes (the other place that compares in EU is Switzerland, but it's much more expensive).
Also, they didn't want to pay for poorer countries that will never get their economy together.
The UK was one of the few net contributors to the EU.
"They didn't want unspecialised poor immigrants from Eastern/Southern Europe" -> the fact is that they were getting a lot of graduated/high specialized immigrants from southern Europe as well.
I guess it is because the folks outside big cities never had a chance to interact with the immigrants and see the value they add to the society, rather than seeing them as a threat. It is quite sad :(
This begs the question: Would it have been as bad as Brexit to restrict worker movements a little more? I would suggest that such restrictions could have even had an overall benefit if done properly. It's just that Germany desperately wanted those cheap workers.
tl;dr: Nobody wants to become a truck driver anymore, even with improving salaries. Higher qualifications and better working conditions might solve the problem in the future.
In the meantime we still need drivers from eastern europe.