Yes. They would have to let all EU citizens live and work(!) in their country.
Including Poles, Romanians, all those people the far right despises.
They wouldn't have to let war refugees and asylum seekers in, though.
Which is exactly the same agreement they have now - except the refugees and asylum seekers, which have recently been regulated, but not implemented yet in most EU countries.
The great majority of foreigners in the UK are coming from non-EU countries, and they are not refugees. Leaving the EU will not change that.
Including immigration? That seemed to be a big factor in the Leave vote.