Sure, but the stored credential is not reusable in another context. Which is not true for the majority of passwords. As a bonus, the authorization is probably also revocable via the external identity provider, although that's more about service-to-service permissions than account compromise.