> At worst case scenario you've literally added no security at all with the pepper.
Yes, I think this is the most likely failure mode (though not necessarily the only one - crypto can fail in very suprising ways!).
But even this is harmful, since you are potentially making changes to security-critical code for no benefit. At best you get more complexity and more chances to introduce bugs, plus a false sense of security.
If the salt remains the same length and you add the pepper on top, it won't make the final hash less secure/strong, due to the way hashing algorithms are folded.
At worst case scenario you've literally added no security at all with the pepper. There's no rational scenario where it reduces the security when all other things remain equal (i.e. you aren't replacing the salt with a pepper, or reducing the salt's length/complexity for the pepper, etc).