Yes, with your open OpenID server you basically become your own SSO. I've set up a Keycloak instance for my self hosted stuff and now I can add 2FA to almost any web self-hosted service without the service even needing to have support for it.

Keycloak is quite a complicated system to configure, though, there are easier alternatives out there. If you're just trying to get anything up and running, something simple like Authelia may be better for your use case (disclaimer: I've never tried it, but it seems light weight and other people online seem to recommend it).