My suspicion is that this system is a cop-out, like when Google blames their screw-ups on "the algorithm".

There probably has been a recent surge of right-wing users and, the Matrix administrators, which probably lean left-wing, have seen that surge as a problem and have created the reputation system as a way to get rid of those users or, at least, to give them the hypothetical yellow badge. And if someone says "censorship", they will blame it on the "downvotes" those users got.