Its support is routinely required and enforced by both clients and servers. Servers won't let clients do AUTH without STARTTLS first. Clients won't send emails if they are set to use it and it's not supported. One can certainly misconfigure the hell out of mail server, but then no "secure from start" will stand a chance against these leet skills either.
Besides, STARTTLS dates back to 2002, so not sure what's up with your "?!?!" and "now we need to ..." comments. Looks like a basic rant to me :-/
Connection to smtp.comcast.net port 587 [tcp/submission] succeeded!
220 resomta-ch2-12v.sys.comcast.net comcast ESMTP server ready
EHLO [2601:b:x:x:x:x:x:x]
250-resomta-ch2-12v.sys.comcast.net hello [2601:b:x:x:x:x:x:x], pleased to meet you
250-HELP
250-AUTH LOGIN PLAIN
250-SIZE 36700160
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-STARTTLS
250 OK
AUTH LOGIN
334 VXNlcm5hbWU6
At least with tcp/465 (SMTPS), the SMTP session (and the potential to send plaintext credentials) won't even start until the TLS session is up. That limits the attack surface to, "Does the client do something stupid with TLS, like not verify the peer's certificate or accept an aNULL/eNULL ciphersuite?"
If the MUA has a "Use SSL?" checkbox in its SMTP settings and it's not checked, then the MUA will send its credentials in the clear on tcp/587 if the server advertises/accepts AUTH LOGIN/PLAIN/CRAM-MD5/etc. before/without SMARTTLS, like Comcast's SMTP servers do. This scenario can't happen with SMTPS.
From our perspective, most people start (or at least should start) with the premise that email is not secure. There are things that we can do to make email more secure, but as an email service provider, we have to balance delivery with security. Most customers would rather have their email delivered to an inbox than us only deliver to an inbox where we can validate the TLS certificate presented by the server. In-fact, delivery is the entire reason companies like Mailgun exist.
That doesn't mean we don't care about security. We've rolled out opportunistic TLS about a year ago, we're working on rolling out mandatory TLS soon (with strict certificate verification), and have a slew of security feature coming down the pipeline that I think (some) customers will be really excited about. However, when it comes to delivery, the truth is that features like mandatory TLS are not really requested or demanded by customers. In the email world, delivery trumps security which is not surprising when you work from the premise that it is not secure to begin with.
Please don't take my comments as a knock on Mailgun. You guys rock.
It was more the frustration with the idea that 1) encryption is optional for certain network protocols, and that 2) you upgrade a plaintext connection to a secure connection. This simply fails in practice, and we have seen this time and again with HTTP/HTTPS.
1) is changing, which is good.
2) is something people keep pushing for because it should work. Afterall you can configure your server to require a STARTTLS, and have all your client's configured to always use STARTTLS and abort out if it doesn't become secure.
Stop that. You are already making this to complicated. Crypto is hard enough as is. (See lower comment re: Comcast and STARTTLS). Ignoring the fact that you are mixing Layer 5/6 concepts into your Layer 7 application prototol, you are creating an insecure-by-default system. Its plaintext, unless you do these things. Stop that. You are doing it backwards. Make it a separate port, and SSL/TLS happens before the application protocol is even accessed. "Is this connection secure" and tracking state and transition into/out of that is not the job of an application protocol. Make this simple and boolean.
Off topic: Another thing I don't get is how do the relays validate each others' certs? You can get a postfix box up and running with a single command, no CA's involved. How does Mailgun know that it's talking to my mail server at mail.example.com? Or is that what DKIM doing? If so, then aren't we just relying on DNS to distribute public key info, which is not secure? Also, what if I don't publish DKIM?
Proper validation can be enabled, of course, but it is typically not by default. Of course, connecting over a "plain text" connection and then "upgrading" to TLS has its own problems, as you pointed out. The most significant is that a man-in-the-middle can simply filter out the "250-STARTTLS" response from the server and the client will not attempt to use encryption.
Unfortunately, in order to be RFC-compliant, an MTA cannot refuse to transfer mail just because encryption (TLS) is not available.
I wholeheartedly agree that the e-mail protocols need updated -- or perhaps even completely overhauled -- but I think it will be a long time before we see that happen. Besides, even if new protocols were introduced tomorrow that fixed all of this, it would be years before there was 100% adoption meaning that we'd still have to support the current protocols until then.
I don't think you can say, fairly, that "this is incredibly stupid and short sighted", though. I've been working professionally in I.T. for nearly 20 years and SMTP was still around way way before that. The problems that we are currently trying to solve simply did not exist then and almost certainly could not have even been imagined that long ago (recall that RFC733 was authored in November 1977 and RFC822 in August 1982).
Overall, SMTP has proven to be a fairly robust protocol but, as with anything, it needs to adapt.
Almost all mail clients (MUAs) I've seen will validate the TLS certificate.
Almost no mail servers (MTAs) will validate the TLS certificate. This is because a) TLS isn't required, so a MITM could just strip the STARTTLS and everything would need to continue to work, and b) It doesn't provide any security.
TLS on the client is really securing the client credentials - not the email. TLS on the server is per hop - not to the final destination, so it doesn't protect a MITM attack.
If you want encrypted email, encrypt the email - don't use per hop encryption.
Right. This is what I was referring to -- sorry, I could have been more clear.
You have to either add an exception (works, but not ideal), or find the exact hostname which will pass Thunderbird's certificate validation (dreamhost uses a wildcard certificate for * .mail.dreamhost.com, so after digging on its wiki you find out that the correct alternative is in the form of subn.mail.dreamhost.com, where n is the same number on the mail server's real hostname, which you cannot use because it has two components where the * can match only one).
(Edit: can't make the asterisks show up correctly.)
(Edit: misread MTA as MUA. Yeah, most MTA's can't validate the certificates.)
If you are running a mail system like Postfix, with DNSSEC and DANE turned on, then your server will refuse to deliver mail to mine unless it can negotiate a TLS encrypted connection, and the certificate used matches the cryptographically signed fingerprint I publish in the DNS.
One can obviously configure his/her own MTA to not send out mail unless encryption is being used (although one would certainly end up with undeliverable mail), but RFC2487 says that you may not refuse to accept mail (again, on 25/TCP) that is not encrypted.
As with DNSSEC and DANE, like you mentioned, one can also require his/her MTA to validate the server's certificate although this isn't typically enabled by default.
We could do with another attempt to redefine email, even if it's unlikely to get deployed. It might be worth considering how to unify email with IM in the process.
One Word: SSLStrip
We have had to completely overhaul HTTP/HTTPS because this "start insecure, go to secure" just does not work. And even then, its been a series of hacks like Strict Transport Security, preloading lists for HSTS into browsers, OSCP Stapling, hard coding revoke lists, etc.
And now we need to go through all this pain again for mail?!?!
This is incredibly stupid and short sighted. Any network protocol that is sending personal/private information, or sends authentication credentials, needs to do so over an encrypted channel. New protocols get a single port, encryption is required. Legacy protocols which do not have an encrypted channel, like SMTP, POP, IMAP, FTP, HTTP, need a separate, dedicated port number for the encrypted version.