To put it another way: suppose I ask someone "why did you forget my birthday but not his?" It's not that I want them to forget his, too (though that is one literal interpretation). It's to suggest that the answer isn't merely "I forget birthdays in general."