My understanding is that email headers can be spoofed and should not be relied on. So this is probably too hard to ask from an email client (i.e. a program that reads and writes emails) that only has access to the email headers. Whereas for the email server (the computer that sends/receives emails for this address) this would be trivial, just keep a list of emails that the server has sent.