I agree with all of this (no wonder since Thomas' opinions and mine largely come from the same place), but dark horses like Kelby Ludwig's canonicalization bug make me even happier if I can outsource that infrastructure to a vendor, perhaps exacerbated since I'm also an infrastructure-focused person.
I'm not all starry-eyed about Cognito, but I know the next libxmlsec1 0day isn't my problem. (Except of course that probably is because some clients don't have Cognito :-))
My best advice for safely using SAML is: use a library that everyone uses and that is well tested. This is a place where I might stand up a separate service in a different language to get a popular SAML library rather than the shady one that is most convenient for your preferred platform.