A counter argument is that if Google doesn’t care enough to manipulate its reviews because the stakes are too low, then they might also turn a blind eye to restaurants, businesses, and other listings manipulating the system themselves (again assuming it’s too small time for Google to care).

But I don’t have a better solution either, my wife and I have followed almost your exact same pattern of relying on Yelp until it became untrustworthy and then Google Reviews (which is what I still use now).