Fully agree. I would even suggest to have separate values for agree/disagree and high/low quality. Flagging might be justified in many cases but unfortunately can be misused (i.e. allows arbitrary suppression of valid opinions).

EDIT: flagging could e.g. increment a visible counter and serve as a trigger to the admin; assuming the admin is a neutral arbiter assessing the compliance of the post with HN and legal requirements he/she could remove the post or comment, and leave a rationale for his/her decision; that could prevent arbitrariness.