postfix (SMTP), dovecot (IMAP), davical (CardDAV for addresses, CalDAV for calendars) have been working well for me for more than five years.

I experienced blacklisting when I switched to a newer dedicated server hardware recently, which meant new IP addresses. The only mail server that didn't accept mails from my new addresses was Hotmail / Outlook.com though. It was fixed after filling in their form and waiting a day. Never had trouble with Gmail.