News for hackers. Stuff that matters.
Re:The “Chad” bug by 0xcde4c3db on Wednesday December 30, 2015 @11:59PM (#10816238)
Besides the usual regex aches and pains, the grammar for email addresses is far more complex than most people realize. According to a highly-voted Stack Overflow answer [1], the current RFC-specified grammar for addresses can't even be matched with regex alone. Combining the edge cases of the grammar with (say) Unicode normalization sounds like a recipe for hours of fun.

[1] https://stackoverflow.com/questions/201323/using-a-regular-e...

Re:The “Chad” bug by eli on Thursday December 31, 2015 @01:25AM (#10816504)
I think many people are thinking about email address validation the wrong way.

RFC 822 describes how messages are encoded when email servers talk to each other. It isn't really about email address validation and is not intended to be used to validate a form field on some registration page.

Unless you're writing an MTA or similar piece of infrastructure there is no reason you should be using the RFC grammar. Even if implementation were easy, it probably isn't what you want. For example, the spec permits inline comments but that's a nonsensical thing to have in the middle of an address you typed into an HTML form. Email addresses entered on a web form should be rejected if they contain comments, IMHO.

I think what most developers really want to know is something like: Can this given email address receive messages? Or: Does this given address actually belong to this user? Well, the only way to test that is to send it a message. At best, regex validation might warn you earlier that a given address couldn't possibly work because it's so obviously malformed. But you can't validate your way into getting people to enter their real email address if they don't want to or if they don't know what it is. If your intent is really just to help catch typos and mistakes, you'd be much better off looking to something like mailcheck [0] which will flag common typos like "[email protected]" even if they result in valid looking addresses.

[0] https://github.com/mailcheck/mailcheck

Re:The “Chad” bug by saurik on Thursday December 31, 2015 @01:46AM (#10816565)
I agree with you "100% in spirit", but feel the need to point out that RFC 5322 (822 is ancient...) is for MIME, and thereby isn't what is even used "when e-mail servers talk to each other": it is the format of the e-mail message itself that is used only within a mailbox (when either your client or, if you are using IMAP, your server parses the message).

The actual standard used by e-mail servers talking to each other is 5321 (which you might still know 821 ;P), the standard for SMTP: this protocol actually has a different way to write comments and escape characters, as it is embedded into a different structure (which I think decimates the arguments people tend to make that you should validate comments).

Years ago I was working quite in earnest on an e-mail server suite, and at the time I was extremely deep in the various standards, and wrote a comment that goes into somewhat more depth on the semantics of e-mail verification. The example I was really happy with is the notion that you would never ask your user to HTML escape their username or password ;P.

https://news.ycombinator.com/item?id=4486872

Re:The “Chad” bug by xenophonf on Thursday December 31, 2015 @01:10PM (#10818789)
I really hate browser- or webapp-side email validation. I've rarely seen a web developer get this right. My advice is to treat email addresses like blobs and attempt delivery with a confirmation link. That will catch users who typo their own address, and it will let people use whatever address format they want (the "+" in an email address not being considered valid is my own personal bugaboo).
Re:The “Chad” bug by jgalt212 on Friday January 01, 2016 @07:28AM (#10821717)
I dunno. I think people with edge case emails (i.e. those not easily validated with a simpleish regex) should be nudged towards getting more standardized emails.

Our company won't accept weird emails for free trial sign ups. We should be nudging users towards good behavior.

Of course, if your email address is provided at the corp level, then you don't have as many options.

Non-standard emails can be used as tools for phishing is another reason why we should not validate them.

Re:The “Chad” bug by huuu on Thursday December 31, 2015 @02:56AM (#10816738)
I very much agree because in the end the only validation seems to be "does it contain a @ character somewhere in the middle".

For example: example@localhost is valid but is refused by most validators.

Re:The “Chad” bug by maweki on Thursday December 31, 2015 @04:40AM (#10816970)
Localhost is obviously not a fully qualified domain name which would be expected in any internet setting.
Re:The “Chad” bug by hcf on Thursday December 31, 2015 @06:25AM (#10817192)
Top level domains do resolve though, email@bs is completely legal to try to send email to.
Re:The “Chad” bug by lmm on Thursday December 31, 2015 @11:58AM (#10818422)
> Email addresses entered on a web form should be rejected if they contain comments, IMHO.

Why? I use several addresses that start something like myfirstname.mylastname@... . A comment at the start could make it much easier for me to use browser autofill. (Arguably that's "really" a browser UX issue, but we work with the tools we have)

Re:The “Chad” bug by kccqzy on Thursday December 31, 2015 @12:21AM (#10816326)
I find that quite unbelievable. When I had a similar problem last year, the first resource I found was a W3C specification[1] about <input type=email>. The specification clearly states that email addresses should match:

    /^[a-zA-Z0-9.!#$%&’*+/=?^_`{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*$/
Since this is an official W3C doc, I see no reason why people shouldn't use this.

Edit: There is also a version by WHATWG[2] here:

    /^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/
It apparently does a more thorough validation than the W3C one in the domain part, but the difference between the two is not apparent in practice.

[1]: http://www.w3.org/TR/html-markup/input.email.html [2]: https://html.spec.whatwg.org/multipage/forms.html#e-mail-sta...

Re:The “Chad” bug by taspeotis on Thursday December 31, 2015 @12:57AM (#10816431)
The .NET Framework's email address validation has a pretty ... comprehensive ... regular expression [1].

[1] http://referencesource.microsoft.com/#System.ComponentModel....

Re:The “Chad” bug by chris_wot on Thursday December 31, 2015 @03:48AM (#10816853)
Holy line of cartoon swear words Batman!

I would hate to be the programmer who had to debug that regex.

Re:The “Chad” bug by leviathan on Thursday December 31, 2015 @04:19AM (#10816923)
You clearly haven't seen the regex that matches emails according to RFC822:

Behold: http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html

Re:The “Chad” bug by chris_wot on Friday January 01, 2016 @11:35AM (#10822273)
/me falls off chair
Re:The “Chad” bug by wyldfire on Thursday December 31, 2015 @12:51AM (#10816406)
>Since this is an official W3C doc, I see no reason why people shouldn't use this.

Well, as far as authority/canon goes, it's typically dictated by the IETF and not W3. And, sure, they could cooperate with one another but, really -- if there's an authority on the protocols that describe email (SMTP, POP, IMAP, etc) -- it shouldn't be the World Wide Web Consortium.

That said, the IETF does tend to draft RFCs that reflect actual implementations (at least their intended design), but since they often bias towards interoperability, it's unlikely they'd narrow the scope of the email address grammar.

Re:The “Chad” bug by seszett on Thursday December 31, 2015 @03:01AM (#10816754)
This forces users who have internationalized email addresses (say whatever@maré-design.fr - just a random IDN I know) to lookup the punycode-encoded version of the domain.

Ditto for internationalized TLDs like something@taiwan.台灣).

I'm not sure if this is really that bad, but I don't like it.

Re:The “Chad” bug by kijin on Thursday December 31, 2015 @03:33AM (#10816820)
It's the job of the validation engine to convert all URLs and email addresses to a canonical form -- either punycode or actual UTF-8, but never both. I prefer standardizing on punycode because it's easier to handle.

Otherwise you're just asking for trouble when the same user who signed up with an email address at maré-design.fr later tries to reset their password with an email address at xn--mar-design-d7a.fr. Sorry, you don't seem to have an account with us.

The same thing happens with URLs. Some browsers send the Host: header in punycode but send the referer in UTF-8. Who knows how they encode CORS headers and all the other newfangled stuff that contains bits and pieces of URLs. You have to consistently convert one to the other before using any of them.

IDNs are a mess.

Re:The “Chad” bug by seszett on Thursday December 31, 2015 @03:43AM (#10816842)
> It's the job of the validation engine to convert all URLs and email addresses to a canonical form

I don't think <input type="email"> does that though. Maybe it should, but users might be surprised to see their address automatically turn into some ugly unreadable xn--whatver-d7a domain. After form submission then yes of course any sane process will convert them to a canonical form.

I know there were good reasons to use punycode instead of UTF8 for IDNs, but it sure is a mess.

Re:The “Chad” bug by kijin on Thursday December 31, 2015 @03:54AM (#10816863)
Yeah, I think we'll have to treat email addresses as just another kind of untrustworthy input that needs to be escaped in various ways depending on the context. Punycode for storage and validation, back to UTF-8 for display, back to punycode when it goes in a mailto: link, etc. Ditto for URLs. The days of easy parsing are sadly forever gone.
Re:The “Chad” bug by oinksoft on Thursday December 31, 2015 @12:52AM (#10816409)
Practically, those W3C/WHATWG regexes probably are fine, but they aren't correct. For instance they fail to match the valid address c."@"[email protected] because they don't handle quoted strings correctly. I'd expect they also match some invalid addresses. My understanding was that only a Perl regex can test an address against the standard: http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html
Re:The “Chad” bug by Dylan16807 on Thursday December 31, 2015 @02:12AM (#10816633)
It may be incorrect to reject that email, but it's good to reject that email.
Re:The “Chad” bug by __david__ on Thursday December 31, 2015 @02:31AM (#10816670)
Why reject a perfectly valid email? Why do you get to thumb your nose at that particular one?
Re:The “Chad” bug by Dylan16807 on Thursday December 31, 2015 @02:50AM (#10816718)
It's only 'valid' because of the particular decisions made by someone writing a random RFC. There's next to no reason to allow quoted strings, and even less reason to allow a different character set inside of them.

Email addresses do not need escape sequences.

Re:The “Chad” bug by kevin_thibedeau on Thursday December 31, 2015 @11:56AM (#10818405)
Where do you draw the line? I occasionally have valid GMail addresses with "+" in them rejected because someone wrote a crappy validator.
Re:The “Chad” bug by Dylan16807 on Thursday December 31, 2015 @05:33PM (#10820089)
Allow all the characters that are supposed to be valid outside of quotation marks. Letters, numbers, .!#$%&'*+-/=?^_`{|}~

That way if you don't mind letting through double periods or domain segments having more than 63 characters, you can validate an email with two character classes separated by an @. The basic regex looks like [xx]+@[yy]+

Re:The “Chad” bug by mikeash on Thursday December 31, 2015 @06:37AM (#10817215)
Doesn't this sort of RFC typically summarize and codify real-world usage, rather than coming up with new stuff? If a feature is in there, I'd have thought it's because somebody uses it.
Re:The “Chad” bug by pc86 on Thursday December 31, 2015 @10:40AM (#10817989)
At some point you just have to reach the level of "Yeah... don't do that." I would suspect have "@" as part of your email probably approaches that line.
Re:The “Chad” bug by mikeash on Thursday December 31, 2015 @11:08AM (#10818156)
It would be interesting to know the context. If this feature was already in a popular piece of software, then saying "don't do that" in your standard risks fragmenting things rather than codifying them, and that's no good either.
Re:The “Chad” bug by immibis on Thursday December 31, 2015 @07:16PM (#10820500)
> It's only 'valid' because of the particular decisions made by someone writing a random RFC.

You know that's exactly what "valid" means, when talking about something standardized, right?

In HTML there's no reason to use &quot; outside of an attribute value. Should browsers reject it?

Re:The “Chad” bug by oh_sigh on Thursday December 31, 2015 @01:10PM (#10818788)
I don't think gmail will let you register that email address.
Re:The “Chad” bug by immibis on Thursday December 31, 2015 @07:17PM (#10820504)
Okay then, c."@"[email protected]
Re:The “Chad” bug by zurn on Thursday December 31, 2015 @12:56AM (#10816421)
The email input regexp is only about matching the atext@domain subpart of the syntax. The full email address parsers are about parsing the full allowed syntax you can put in an email to: field that includes address lists and display names.

Also W3C produced HTML specs are hardly gospel about email-related things (but I don't know whether there's anything wrong in this case).

Re:The “Chad” bug by zurn on Thursday December 31, 2015 @07:43AM (#10817374)
Replying to myself (can't edit anymore), looks like the W3C spec bungled it in your [1] by referencing "atext" from the RFC. For example, as specified it disallows addresses with periods in them. In the RFC, "atext" is just a building block in the syntax of the local-part before @ sign.
Re:The “Chad” bug by kccqzy on Thursday December 31, 2015 @06:33AM (#10817206)
But the question is, if a signup form is asking for your email address, will you type in your name, a pair of angle brackets, and then your email address in between? That's simply not what the web developer wants here.
Re:The “Chad” bug by lmm on Thursday December 31, 2015 @12:01PM (#10818431)
Will I type that in? No. Will I copy-paste my email from my address book program? Quite possibly.
Re:The “Chad” bug by StavrosK on Thursday December 31, 2015 @10:33AM (#10817933)
Saying "it can't even be matched with a regex" is trivial. Here's another, very simple thing that can't be matched with a regex: Matching pairs of nested parentheses.
Re:The “Chad” bug by AceJohnny2 on Thursday December 31, 2015 @12:53AM (#10816411)
The spec of what HTML will accept for an email in an input form is quite different from what email (which predates HTML by a lot, the major RFC 822 being from 1982) actually accepts.
Re:The “Chad” bug by djur on Thursday December 31, 2015 @12:59AM (#10816434)
The W3C and WHATWG are web standards bodies. Email isn't in their bailiwick. The acceptable format for email addresses is defined in IETF RFCs.
Re:The “Chad” bug by hitekker on Thursday December 31, 2015 @01:06AM (#10816456)
direct link to image visualizing the complexity of the state machine required: https://i.stack.imgur.com/SrUwP.png