It is governed by the Privileges and Immunities clause under the authority of the States (not Congress) and has been judicially recognized as a fundamental right tied to that same clause since at least 1823.

It was considered so fundamental during the drafting process that there wasn’t a need to enumerate “freedom of movement” specifically.