It was always more paternal than fraternal, quite a lot of the freedom was won with violence (e.g. Kenya), if it were equal there would be no talk of "carry the idea alone", the UK did not join the EU in 1973 it joined the EEC (EU free movement only appeared in 1992), and the British public made it clear that they were only OK with free movement so long as the number of nonwhite people in the UK remained small.
1971 and the Kenyan Asians was the trigger for the https://en.wikipedia.org/wiki/Immigration_Act_1971 - that was the point at which Commonwealth citizens lost the right to live in the UK itself. Two years before EEC accession. (And the same year as decimalisation of the currency!)
At the height of the Empire there were effectively no internal movement restrictions other than income inequality and the difficulties of transport; you no more needed permission to move from Delhi to London than from Glasgow.
Oh and gave it a go as in gave letting everyone in the old empire continue to be able to come to UK a go.