I don't think you're really correct about "email addresses" being context-free, or at least, citation, please?
When I look at a generic "email address" entry field on a random form on the Internet, say on the sign-up page for some hot new startup's service, I expect it to take what RFC 5322 §3.4.1[1] calls an `addr-spec`; specifically, I don't ever expect such fields to take the grammar of what that RFC calls an `address`. I don't think most people are going to think they can enter that, nor would most programmers even think to implement it. And I certainly wouldn't want to try explaining it to a PM…
If you accept that assumption, what about `addr-spec` isn't regular?
Also, using that assumption, your "perfectly valud email addresses such as …" would appear to not be valid, as it has unbalanced quotes. (In fact, even under the grammar of `address`, I'm not sure it's valid; it feels like it should be invalid for the same reason, but I've not rigorously checked this.)
> When I look at a generic "email address" entry field on a random form on the Internet, say on the sign-up page for some hot new startup's service, I expect it to take what RFC 5322 §3.4.1[1] calls an `addr-spec`; specifically, I don't ever expect such fields to take the grammar of what that RFC calls an `address`.
Well, sure. Let's look at what RFC 5322 defines as an addr-spec[1]:
addr-spec = local-part "@" domain
And how does it define a local-part? local-part = dot-atom / quoted-string / obs-local-part
Let's ignore quoted-string and obs-local-part for the moment. What is a dot-atom? dot-atom = [CFWS] dot-atom-text [CFWS]
And what is CFWS? CFWS = (1*([FWS] comment) [FWS]) / FWS
What's a comment? comment = "(" *([FWS] ccontent) [FWS] ")"
So far, all of this has been matchable with a regular expression. But what's a ccontent? ccontent = ctext / quoted-pair / comment
See that there? A comment is composed of a balanced pair of parentheses around, perhaps, another comment! Thus (this (is (a (heavily (commented (email \(address))))))[email protected](some more (to prove (the point))) is a perfectly viable RFC5322 address!Pair-balancing, of course, is impossible with regular expressions, since matching pairs requires push-down automata (which match CFGs) and cannot be done with finite-state machines, (which match regular expressions).
QED.
> Also, using that assumption, your "perfectly valud[sic] email addresses such as …" would appear to not be valid, as it has unbalanced quotes.
Nope, there are no unbalanced quotes in (this)"()<>[]:,;@\\\"!#$%&'-/=?^_`{}| ~.a"(is)@(valid)example.org(honest): the first quote balances with the third, while the second quote is one of a quoted pair \" (which is allowed within a quoted-string, which is allowed within a local-part). It's all allowed per the spec.
I'll admit that it's a bit surprising, but it's true. One simply cannot match a valid RFC5322 addr-spec with a regular expression. One can, of course, match it with something which pretends to be regular but isn't really (as I noted).
Well done.
You can find multiple short email validation snippets using Lua LPeg pretty easily, but this is from the Lua creator's talk about LPeg which includes a part about proper RFC822 validation and how complicated it is for regex, but can be concisely done with PEGs.
http://program-transformation.org/pub/WGLD/Austin2012/Robert...
Here is the entire PEG slide. (Looks CFG-ish to me.)
address <- mailbox / group
group <- phrase ":" mailboxes? ";"
phrase <- word ("," word?)*
mailboxes <- mailbox ("," mailbox?)*
mailbox <- addr_spec / phrase route_addr
route_addr <- "<" route? addr_spec ">"
route <- ("@" domain) ("," ("@" domain)?)* ":"
addr_spec <- local_part "@" domain
local_part <- word ("." word)*
domain <- sub_domain ("." sub_domain)*
sub_domain <- domain_ref / domain_literal
domain_ref <- atom
domain_literal <- "[" ([^][] / "\" .)* "]"
word <- atom / quoted_string
atom <- [^] %c()<>@,;:\".[]+
quoted_string <- '"' ([^"\%nl] / "\" .)* '"'Are those parens really permitted in the host name? I'm looking at RFC 3696.
> 3. Restrictions on email addresses
> .... The syntax of the domain part corresponds to that in the previous section.
and
> 2. Restrictions on domain (DNS) names
> Any characters, or combination of bits (as octets), are permitted in DNS names. However, there is a preferred form that is required by most applications. This preferred form has been the only one permitted in the names of top-level domains, or TLDs. .... The LDH rule, as updated, provides that the labels (words or strings separated by periods) that make up a domain name must consist of only the ASCII [ASCII] alphabetic and numeric characters, plus the hyphen. No other symbols or punctuation characters are permitted, nor is blank space.
> Are those parens really permitted in the host name? I'm looking at RFC 3696.
RFC5322 states that an addr-spec consists of a local-part, followed by @, followed by a domain. It states that a domain may be a domain-literal; a domain-literal may begin and end with commented folding whitespace (CFWS). It's all in https://tools.ietf.org/html/rfc5322#section-3.4.1
It's only the dtext portion of the domain-literal which must consist of printable ASCII characters, not including [, ] or \.
Yes, foo@bar!baz!quux is a valid RFC5322 email address (decimal 33, i.e. !) is allowed, per the dtext production of the spec) — and so is foo@(comment (nested comment (escaped \(comment)))bar!baz!quux(another (comment (to prove) the point))
Whether one should ever actually use such an address is, of course, another matter entirely.
The possibility of having a valid email address where the second-level domain component can't actually be registered as a domain, to the best of my understanding, eg "bar!.com", is interesting.
It is! Conceivably, it could be used to implement a new kind of domain-less mail system, e.g. foo@$megamail or something.
If it doesn't, remind me to patch it to not accept that.
I have strong views on email validation, and they include telling people who use that kind of address to go register on someone else's site.
You'd be wrong to do so. The whole point of RFCs and Standards is to take things out of the realm of personal preference.
Also, I suspect that a validator which allowed reasonable addresses like jim(somesite)@foo.invalid (which is both a good use of comments and what the + hack emulates) or "Ted Smith"@bar.invalid or "work@home"@jobs.invalid or "William \"Bill\" Jones"@baz.invalid but disallowed unreasonable ones would also be able to solve the halting problem. How many perfectly reasonable allowed features must one use before it's considered abuse?
> I have strong views on email validation, and they include telling people who use that kind of address to go register on someone else's site.
An email address is, simply, an address which can be given to an Internet mail server in order for it to properly route an email. All of the addresses I gave are perfectly valid ways to do so. An addresses is a property of the addressee, not of the addressor.
In other words: it may be your site, but it's my address.
Many of the more arcane things that can technically be done in an email address seem to me to be in the "turned out to be bad ideas" bin, and I have no problem making them be de facto deprecated even if no RFC has yet caught up to that.
Also, no halting-problem issues at all; my preferred approach would be to disallow certain classes of characters.
That may be true, but it's something else entirely for someone to drop them because the problems they cause … were enabled by him. An email address using characters you don't like is perfectly valid, and perfectly deliverable, right up until you refuse to deliver it. It doesn't cause any problems until you decide to make it cause a problem.
> Many of the more arcane things that can technically be done in an email address seem to me to be in the "turned out to be bad ideas" bin, and I have no problem making them be de facto deprecated even if no RFC has yet caught up to that.
> Also, no halting-problem issues at all; my preferred approach would be to disallow certain class
I really don't see why you think it's a bad idea to disallow more than alphanumeric (and plus? and ampersand?) characters in the local-part. Cui malo?
I prefer this because I work in a world where email addresses are provided to me, and possibly stored and then retrieved and used and even displayed, as text, and the RFCs allow some nightmarish things when you consider the interaction of the syntax the RFCs permit and the set of characters which are sensitive to one or more of the non-MTA components of that chain.
Your original example, for instance, contains characters that require escaping or at least careful handling for multiple situations, and even goes so far as to contain things that will be interpreted as escape sequences in some contexts. I'm sure that Robert(';DROP TABLE users;)(<script type="text/javascript">alert("Bobby Tables")</script>)@not.a.hacker(honest) will be terribly disappointed to know that he needs to use a different address to register on my site. I'll send him over to you instead.
Technically-correct is the best kind of correct:-)
But I do think it's important to note, since there really are differences between regular and 'regular' expressions.
I wish that were the case. http://www.regular-expressions.info/refunicode.html
(They used to have a much more useful and concise comparison table but I can't for the life of me find it.)
I guess what I really meant is that programmers are talking about PCRE in terms of power, not in terms of exact syntax. In particular, they have recursive patterns, which are sufficient to pull them up to context-free grammars.
>>> from django.core.validators import EmailValidator
>>> EmailValidator()(""""()<>[]:,;@\\\"!#$%&'-/=?^_`{}| ~.a"(is)@(valid)example.org(honest)""")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/django/core/validators.py", line 203, in __call__
raise ValidationError(self.message, code=self.code)
django.core.exceptions.ValidationError
If it is a regular expression, then it is not able to match all valid email addresses, because the grammar of email addresses is context-free, and regular expressions can only match regular grammars. It doesn't matter if it is defined or not: if it's a true regular expression, then it simply cannot validate email addresses.
(it may, of course, be a context-free expression masquerading as a regular expression)
I wonder if the django validator will choke on perfectly valid email addresses such as (this)"()<>[]:,;@\\\"!#$%&'-/=?^_`{}| ~.a"(is)@(valid)example.org(honest)
I suspect that it will, but of course I could be wrong.