Perhaps true in many people's minds, but it largely seems to be the case that native Brits do not want to do those jobs. Either Brits tend to see themselves as "above that", or there is a serious shortage of workers in those areas (structural problem). Therefore, such jobs were left undone or severely delayed. The UK got what it literally asked for and is now finding out that it didn't actually want that.
Brexit was also partly fear of Muslims, not just job competition with Eastern Europeans. You see this in the internal discussion around Syrian refugees from people like Nigel Farage, Douglas Murray and Sargon of Akkad. They wanted to exit the system that enabled that. This particular motive was collectivist.
In addition, Brexit was partly the typical arrogance and wounded ego you see from declining and fallen empires. The glory days are still fresh in people's minds. There's a feeling that Britain is on the decline. People couldn't accept the egalitarian terms and low status of being just another ordinary EU member state. This also is collectivist through and through.
In the Brexit case, it was white Eastern Europeans coming in to UK and taking over low-paying jobs. So, neither a threat of terrorism nor racial supremacy (they're all white). I guess one could be afaid of some regions losing its inherent British culture though.