SAML2 is kind of hairy, technically, but--speaking as someone who used to do this full-time up until six years ago and has stepped into other sings since--most of the pain was from half-baked implementations and utter crap-tier error/exception reporting. Like, on one hand you'd be plagued by a lot of small-time implementations that had a very inadequate appreciation of the spec, and/or only supported about 10% of it, or had fatal flaws in their naive handling of XML or even worse fatal flaws related to XML-DSIG and canonicalization... and on the other hand you'd get these monstrous enterprise products that specialized in offering only the most oblique of views into their inner configuration or error states. It really takes a lot of patience and expert level skills in 5-6 different things to cut to the heart of most practical issues in the space.

SAML is its own nightmare on top of the OAuth/OIDC mess. You're probably in the worst of all worlds right now.