It was only google, only non-response emails, so it is a google specific thing and not general IP reputation.
Switching to fastmail, otherwise exactly the same (domain, equiv dkim and spf setup) solved everything.
Sure looks like I was on the scoring edge, with fastmail being better (or some special trust arrangement) than my previous provider. Which is fine if I had gotten feedback and had a way to resolve this - but practically I didn’t.
Edit: I had a catchall on that domain forward to an account on gmail. It was suggested to me that may have been the problem. But i still do from fastmail, and everything is fine, so it’s more likely that overall reputation or special fastmail arrangements were the issue. Either way, i’m far from being alone with this issue - as evidenced even i this subthread. It is federated but not practically open to everyone, which is an implicit attribute most people associate with decentralization.
Being small may not have been their problem. I run a single-user server and haven't had any trouble with delivery to Gmail.