I run my own mail server, I can tell you this: Buffer overflows and format string vulns in your imap implementation will be the least of your worries.

It is certainly still valuable work to audit the software used for the mailserver. But the challenges lie elsewhere. The biggest chunks are configuration issues and - if you have users that aren't very security savvy - stolen passwords and subsequent abuse of mail accounts for spam.