Sorry, but history is overwhelmingly against you. The historical precedents are legion. We're not talking about some "weird slippery slope argument." This is well established human nature. In fact, many of the items in the Bill of Rights are based around this principle.

In fact, the principle has a much shorter, snappier version: Power Corrupts.