Support for unlimited EU immigration is not 50%, it's much lower. You're thinking of people who voted to Remain, but a significant fraction of those people were motivated by the fear of "punishment" either from Cameron/Osborne's huge tax rises or the supposed EU allies, not actual firm support of EU policies.

A breakdown is available here:

https://dominiccummings.files.wordpress.com/2016/11/screensh...