I'm not so sure about that. E.g. there aren't any German truck drivers and trucks anymore, just companies and drivers from eastern europe, because it's cheaper. Similarly for lots of jobs in agriculture, construction and some industries like food processing. That's surprisingly not a problem because of unemployment in Germany as much as it is a social and legal problem: Eastern European workers abroad are underpaid (minimum wage circumvented), uninsured (social security dues circumvented), without appropriate workplace protections (truck drivers doing overtime, illegally having to sleep in their trucks for weeks on end, lax technical safety of the trucks), and draining the economy of their home countries of workers because factories that could have been nearby are in Germany instead.

This begs the question: Would it have been as bad as Brexit to restrict worker movements a little more? I would suggest that such restrictions could have even had an overall benefit if done properly. It's just that Germany desperately wanted those cheap workers.