Not necessarily, if they hack into one system then getting into another isn't automatic. If the passwords are in a separate filesystem/database than the key, and linked only with software, then unless it's the software that's comprised it would still increase the difficulty of getting both the password and key significantly. It also prevents trivial browsing of passwords via sql commands by rouge employees.