I think there's actually a big difference between morality and truth, and what you're saying applies to truth and not morality. If you went around telling people about germ theory, you could say, I'm objectively right and you're objectively wrong, here's the proof. The truth is the truth; it's an invariant. It's arrived at through discovery and investigation. Morality is about people fighting to get what they want. The "right answer" is arrived at through conflict and seeing who wins. We live in a society where "slavery is bad" won.

When you say that we should ban debate of slavery because it's settled, what you're in effect saying is that society has made a decision and you would like that decision to be final. In this case, so would I. But there's nothing final about it; it's not like God has spoken on the matter. Deciding to ban further appeal of the matter is just a form of moral entrepreneurship, not some kind of absolute correct answer.