The "Caddy of mail servers" should be a plugin to Caddy, since server types are pluggable. Caddy's HTTP, DNS, and TCP+UDP servers are all just plugins.

I typically don't recommend someone running their own mail server, but if they do, I wish there'd be a good one they can use, and that one would definitely be a Caddy plugin.