You can't be a member of the EU (or the EAA if i remember) and restrict immigration from other EU countries.

When the UK finally leaves then yes that could be how immigration is managed.