In England at least, there seems to be a significant “EU immigrants took our jobs” factor as well. As I understand it, that perception was a significant factor in the Brexit vote.

For example see: https://www.theguardian.com/politics/2016/may/20/reality-che...