The single best improvement you could make right now is to ban a user based on IP if they exceed a certain amount of downvotes over a period of time. That would at least fix the spam problem.