Your manager can say "based on feedback everyone says you suck" and you can't really do a whole lot to look into it. You don't know where the feedback is coming from, or if it's even valid.
At a smaller company a manager seems more responsible to look at performance democratically. So if they have a personal vendetta and they fired someone for it, it's much easier for other employees and other managers to see what's going on and call it out (so they might not act on it to begin with).
Of course that's not universal, sometimes at a small company you might have one person with consolidated power so it doesn't really matter (and can be much worse). And of course, at a smaller company peer pressure can keep issues under wraps because you're worried about everyone knowing that you're the one bringing it up.
So yeah, a performance review process needs to exist at larger companies because it makes sense logistically... but criticism of that process doesn't mean it should be thrown out entirely, it just welcomes the discussion about how it can be improved. There are middle grounds between strict performance reviews and small-company democracy.
First, nobody has an incentive to provide negative feedback except for serious cases (i.e. you cannot work with this person) or for political reasons. It's a prisoners dilemma with indefinite iterations, so each player's incentive is to provide positive feedback.
Second, in my experience, it's only reviewed by one person rather than by a management team, which opens it up to an array of biases on the manager's part. If a manager thinks that someone is a great team-player, and a reviewer says that that person is a petty tyrant, they're likely to doubt the reviewer's motives rather than updating their beliefs.
Peers supply the feedback, but one manager has the time/resources to review it... so basically they can interpret it however they decide.
This has the same result as the formal review system, but with a lot less overhead. More importantly, if you're denied promotion out of spite, you know that - they can't just say "well, it's not my fault, it's just how the process worked out".
Yes, good managers can definitely help cut through the BS even with a process in place, it takes time and effort for them - time and effort that could be used on something more valuable. And sometimes, no amount of effort is sufficient to buck the process.
From my experience, by the way, a formal competitive review system (buckets etc) makes it more important to have a good manager, because you absolutely need your manager to play the game well against other managers to yield good results for you, no matter how hard you work. If your manager is indifferent, or can't play the game well, they'll take away whatever was left on the table after the rest, and that's what you'll be stuck with.
My experience has been that the biggest factor is your direct manager. If your manager hates you, then you have a serious problem, because no matter what kind of process (or lack of process) your company has, they can get you fired. On the other hand, a good manager can help you do your best despite any existing "cookie-cutter" process.