The problem is your outbound mail server does not have a good reputation. You also probably do not have a large enough swath of IP space to prevent bad neighbors from becoming your problem. Lots of tricks to getting your email sent reliably to most mail servers. What got me out of hosting mail was trying to send an email to a potential lead I met at a local meetup. His email was hosted at some very small and relatively unknown university, but my email was flat out rejected. Something about that moment just clicked that it's not worth my time to chase down the admins and resolve the problem. I'll just start shunting my email through O365. I still selfhost everything else I can, but I offload mail management to a provider

It's more than that. I'm on a mailing list for a social organization that I've belonged to for 20 years, and has been hosted at the same place for all of that time. I have marked hundreds of the messages not spam, and still Google sends about 80% of them to spam.

Google does not care even if you have regular correspondence with an address -- if the server isn't big enough, it's going to spam. The user's wishes or ideas about what is or isn't spam are irrelevant.

I'm surprised that you didn't just whitelist the email after the repeated ham events...!

I'm guessing dreamhost has a decent chunk of IP space...

All of which they’ve rented out to multiple spammers over the years…

That does seem to be the problem

As a counterpoint, I host a couple of domains with Dreamhost and have had no such issues. They've been serving the email for my main domain for 15 years and AFAIK I've never had any of those emails go directly to spam within Gmail or elsewhere. I do periodic smoke testing by sending test emails to different place, plus I have a reasonable number of external users making use of the domains for email as well. FWIW I moved all my DNS records to Cloudflare, but that was only in last couple of years.

I can pick any ISP today, including Dreamhost, and have clean email reputation and sending ability with FAANG and Microsoft in less than 30 days.