That's a reason I use different sets of SSH keys from different hosts, or at least different classes of hosts.

It's rather troubling to find that many SSH management tools (including Puppet/Chef recipes) deal poorly with multiple keys per user.