It's usually the case that it's cheap to disagree. It would be more beneficial to the review process to have those with dissenting opinions lay out a logical case presenting their concerns rather than enable them to do quicker, drive-by review.

Haven't you had feedback along the lines of, "but did you take into consideration what impact the heat death of the universe will have on your design?" ?

No, it's because downvotes actively censor unpopular opinions based on the fact that they're unpopular rather than their merit. Check out reddit some time and you'll see this happen a lot.