Indeed. The tests tell you that you are breaking a contract that you once thought was important. It might not be any more, but good tests reflect good contracts. Removing the test requires you to think whether any other code expects this contract.