That gives me an idea. Create a decorator or otherwise wrap the logging function as you build the apps test context, and feed it a list of sensitive strings you want to detect. Then each time as logger is called have it assert that all those strings are not within the log message.

This way it would blow up on the test that is leaking the credential so you could track it right down and it would transparently apply to all current and future unit tests without any more effort.