It wasn't? It sure was the point that got the most coverage, at least here on the mainland. Other than "the EU is corrupt because we don't control it", what were the Leave side's arguments?
Freedom of movement seemed like the main issue, well "immigrants" really. It could also be a problem for people voting to remain. Indeed we weren't in the Schengen Area say if you wanted less movement within Europe then Remain could have been the right vote.
The other big issue was the money we pay to Europe which was exaggerated; has the falling pound wiped out that 'saving' yet?
50% of the UK is flat out opposed to freedom of movement. In fact I'd wager it's even less than that since freedom of movement wasn't the only talking point of Leave.