Where does JSON even need to come into it?
Don't get me wrong, I'm not a webdev guy. I have no front end/fullstack experience. I'm a longterm sysadmin/DevOps person. But if you're a power user, you want control over your mail interface/workflow. In that case, all your mailserver should be doing is accepting email for you, storing and indexing it, and serving it via API to clients you're using. I like IMAP, but it doesn't easily support some Gmail conventions (multiple labels per message).
IMAP and SMTP could easily be condensed into an XML/JSON API that could be done over HTTPS; I'm not familiar enough with CalDAV to say that though.
"nvlope gives you full control over your mail with a full-blown JSON API. You can build an email client with just 3 API calls - threading included!"