Yes, that is exactly right, and it's also why certs are short-lived (hours). When certs are renewed, the site the user plans on logging into is not revealed to the Identity Provider.