Git does have tools that allow you to rewrite history to fix situations like this, but by far the easiest solution is to invalidate those credentials so they become worthless.