Potentially, but if you wrote a test for your incorrect code wouldn't you do the same?

In the end I think the important part is not giving tests more trust than they deserve. If you fix the incorrect code and break a test you should give your change a closer look, but also give the test a closer look before you decide which needs to be fixed.

Depends on how you write tests. If you're writing tests by looking at the code you wrote and just asserting that it behaves as written, then yeah. If you're writing tests that assert that your code conforms to a predefined specification, then the tests could be the thing that helps you uncover the bugs.