In my experience people tend to be zealots only in certain very specific areas and quite reasonable everywhere else. I.e. your proposal's running the serious risk of muting and awful loot of folks here on hn just because the might get a bit hot-tempered on specific subjects.
Apple zealots
Mostly rational people who prefer iOS
Mostly rational people who like both
Mostly rational people who prefer Google
Android zealots
There should be some way algorithmically to determine the people who are in the extreme categories, say...
extremism quotient:= (|upv| + |dwnv|) / max (1,|net score|)
Then if that exceeds some threshold over a certain period the person's votes could be ignored or diluted.