I had this exact same problem 3ish years ago and bailed out to Fastmail for the same reason. The thing that made me throw in the towel completely was what I found out from someone who looked into it for me after I shook the tree of my professional and social contacts. This person was involved in Gmail but not directly in anti-spam, but told me that my domain had "limited reputation."
That domain, which predates the existence of Google by at least a year, had been hosted on the same IP address (IPv4 and IPv6) for almost a decade, with the same MX, A, and PTR records for the entire time. Nothing at all changed about how that domain was configured. Yet it was intermittently being flagged as "limited reputation" and either dumped in the spam folder or simply accepted for delivery and silently dropped.
I took that domain and moved it to Fastmail 3 years and 3 months ago--I know the exact date because I paid for three years of e-mail service at the time and recently renewed it--and haven't had a problem since because, unlike my single-server operation that used to be considered an equal peer on the Internet but not any more, Fastmail has enough pull and reputation to not have messages from its subscribers blocked by other e-mail hosts.
I don't think it's necessarily malicious in nature - more the product of spam filtering.
I wonder if email authentication methods like DKIM/SPF/etc.. help this problem these days.
Honestly, it's a huge relief. Self-hosting has gotten much more complicated over the years. It's very nice to know that there's a round-the-clock staff of professionals taking care of security, deliverability, and fighting spam. And software upgrades, of course!