I just don't get it, how hard it could be? How expensive this could be? Because lots of times they just pay these damages to the customer, because no one knows how this very secure credit card data was compromised. This baffles me. Someone, please enlighten us, there must be a valid reason - at least from an angle.

Having a bunch of different domains can serve multiple purposes.

In GitHub's case, they already have githubusercontent.com to avoid serving untrusted stuff from their own github.com domain.

Sending marketing or security scanner (potentially very spammy) notification emails from separate domains can help with reputation too, to avoid your main domain getting marked as spam.

These are all legit; Amex having 20 different of domains, half of which smell like phishing, and still sending emails from other domains is just incompetence. Something like marketing people or someone dealing with strategy deciding to do stuff in a certain way, with nobody technical in the room to tell them why that would be a problem. As an example, a friend of mine's organisation wanted to do a SaaS website for their niche, and a separate website to advertise the SaaS (separate domain, visual identity, everything).

My theory for most of these cases: they would need permission from who knows what department(s) to set up a subdomain of the main domain for their project, and it's easier to just purchase a new domain for the team/project.

Nailed it. This is 100% pragmatism/convenience-based decision making rooted in terrible culture, red tape, bad communication and dumb org charts.

Keep your SPF simple. Otherwise, make sure it works. Aaand, how many people actively monitor their DNS infrastructure?