But at the same time, the consequence for access of account information for cat pictures site on the site holding financial information shouldn't be discounted (passwords, social engineering info etc).

To continue the (somewhat stretched) building analogy my house falling down doesn't affect yours, but someone stealing my access codes from my poorly secured building potentially could affect the security of yours.