My point is that judicial review does not break down simply to determining whether a "privilege" or a "right" is being curtailed. Certainly, if a "right" (and this means specific rights such as those guaranteed by the Due Process clause) is being curtailed, this will tend to push the standard of review higher. But that is not the only factor at work, and there is a very specific meaning of the word "right" in this context.
Also the court is so inconsistent on what gets what and whether rational basis means "we can imagine a rational basis" or "we have evidence for a rational basis" and whether a right gets strict scrutiny (free speech) or not...
For example, it is very difficult figuring out whether circumcision bans would be held to rational basis or strict scrutiny when it comes to religious liberty (it does with regard to eduction apparently, per Yoder v. Wisconsin, not regarding to drug laws absent the RFRA (see Smith v. Employment Department of Oregon), but it does reach regarding bans on ritual slaughter of animals, (see Lakumu Babalu Aye v. City of Hialeah).
But that just illustrates that making some economically-mandatory action into a privilege make the license to that action into a powerful instrument of control. Hence, the No-Fly List, nominally For Your Safety, but mostly To Control You.