News for hackers. Stuff that matters.
Re:Let’s talk about usernames by bradleybuda on Monday February 12, 2018 @12:06AM (#16356673)
> So if you’re enforcing unique email addresses, or using email addresses as a user identifier, you need to be aware of this and you probably need to strip all dot characters from the local-part, along with + and any text after it, before doing your uniqueness check. Currently django-registration doesn’t do this, but I have plans to add it in the 3.x series.

Please don't "normalize" email addresses like this. Not all mail systems are Gmail, and many do treat "[email protected]" and "[email protected]" as different identities. And even if we are talking about Gmail - it's not your identity system's job to deduplicate different logical addresses for the same physical inbox.

Re:Let’s talk about usernames by jcranmer on Monday February 12, 2018 @12:17AM (#16356708)
To emphasize your point: don't touch email addresses. You can get away with doing equality checks on NFKC case folding, but don't assume that you can store a lowercased email address and have it work properly.

Lots of weird email systems exist. Don't assume that everybody works like gmail. And do test that things work right with uppercase letters in email addresses: I've been locked out of systems before because I use an uppercase letter in my email address and one half of the system was trying to match the lower cased version to the actual text.

Re:Let’s talk about usernames by inopinatus on Monday February 12, 2018 @01:50AM (#16357008)
The local part (left of the @) of email addresses is case sensitive by definition and this should not be controversial.

RFC5321 s2.4:

   The local-part of a mailbox MUST BE treated as case sensitive.
   Therefore, SMTP implementations MUST take care to preserve the case
   of mailbox local-parts.  In particular, for some hosts, the user
   "smith" is different from the user "Smith".
Even though it goes on to say:

   However, exploiting the case sensitivity of mailbox local-parts impedes
   interoperability and is discouraged.
This doesn't prohibit a local delivery agent (such as Gmail) consolidating multiple variants into one mailbox, but everything up until that point must refrain from making assumptions.
Re:Let’s talk about usernames by craigds on Monday February 12, 2018 @02:46AM (#16357182)
That's definitely controversial, and in my experience, just plain wrong. Many, many users sign up to a service with [email protected] and get confused when they later try to log in with [email protected]. we had to add a fix for this in our django user model a couple years back. We havent had any complaints with the new, case insensitive system. IMHO this sort of this should be upstreamed but it seems Django have decided to stick with the spec in spite of usability.
Re:Let’s talk about usernames by jcranmer on Monday February 12, 2018 @09:19AM (#16358674)
As you clarify later, both case-sensitive and case-insensitive are wrong. What you generally want is case-preserving: store the original case, but allow case-insensitive matches to select it.
Re:Let’s talk about usernames by chatmasta on Monday February 12, 2018 @03:57AM (#16357432)
This is an especially common problem when entering email addresses on mobile. Often autocorrect will capitalize the first letter. I always make sure it’s lowercase when registering, but I can see how this could easily lead to confusion.
Re:Let’s talk about usernames by lsaferite on Monday February 12, 2018 @01:40PM (#16360799)
Web pages (and apps) that don't indicate the input box is actually an email address so auto-correct won't capitalize for you are a serious peev of mine.
Re:Let’s talk about usernames by craigds on Monday February 12, 2018 @02:52AM (#16357199)
Oh also, we used a CITEXT field for storing email addresses. Store the case the user registers with, and use that for sending comms etc, but uniquify and match (on login etc) in a case insensitive way.
Re:Let’s talk about usernames by pluma on Monday February 12, 2018 @04:20AM (#16357508)
I ran into this problem as well and in the end added a separate field for the normalized (lowercase) form with a uniqueness check on that. Users would still be presented the original case and it would be preserved when sending e-mails but for identification/uniqueness purposes e-mails were considered case insensitive.
Re:Let’s talk about usernames by papito on Saturday February 17, 2018 @08:10AM (#16400516)
Right, something like "email_lc", and "username_lc". This would be your unique index.
Re:Let’s talk about usernames by ubernostrum on Monday February 12, 2018 @12:47AM (#16356805)
Unfortunately, Gmail rules the world and has trained people to expect that they can be sloppy and inconsistent and enter their email address as 'johndoe' or 'john.doe' or 'JohnDoe' or 'John.Doe'... etc. and it just works, I don't understand why your site is broken, because my email works!

And when that happens, trying to patiently pull a "well, technically" and explain to them about RFC this and the specs say that is a way to lose users.

(I actually have extremely strong feelings about email, email addresses and the whole associated mess of specs, but had to tone it down for this article since it was mostly about the various traps you can wander into from naïvely thinking that you can just read a spec or implement something obvious and get away with it)

Re:Let’s talk about usernames by kqr on Monday February 12, 2018 @01:07AM (#16356883)
> And when that happens, trying to patiently pull a "well, technically" and explain to them about RFC this and the specs say that is a way to lose users.

Not in my experience. Showing compassion and agreeing with them that what happened is terrible and you wish things were different but you didn't call these shots back in the days and if they want improvement you can both go together and complain to google, the service which is actually broken.

Most people just want to be listened to. If you can do that, you'll earn a loyal fan, even if you don't do exactly what they tell you to when agitated. Some will even appreciate learning more about the email systems after the fact. They may even get the feeling you went above and beyond by offering to help them with matters outside of your site.

Re:Let’s talk about usernames by ubernostrum on Monday February 12, 2018 @01:47AM (#16356996)
Most people just want to be listened to.

Many people, even after being listened to, and even after having things patiently explained to them, still continue to enter someone else's email address into forms which will send sensitive information to that email address, and complain that they never got their important email, or that some "hacker" has "hacked" "their" email, etc.

In a perfect world this would not happen. We don't live in a perfect world and are unlikely to live in a perfect world any time soon, so we should not be asking "how can we be pedantic and tell users it's their fault for not reading the RFCs", we should be asking "how can we protect users from their ignorance of the RFCs".

(when I wrote this article, I did not expect that this would be the single most controversial line in it from HN's perspective, but I guess by this point I should have anticipated it)

Re:Let’s talk about usernames by jorams on Monday February 12, 2018 @02:22AM (#16357095)
> has trained people to expect that they can be sloppy and inconsistent and enter their email address as 'johndoe' or 'john.doe'

Has it? In my experience, even if people know dots and the part after the + don't matter for their Gmail account (and most don't) they know it doesn't matter to Gmail, a peculiarity they can use to create multiple accounts on a single website without creating new email accounts.

(This is different from case insensitivity, which the majority of popular email providers seem to implement.)

Re:Let’s talk about usernames by spc476 on Monday February 12, 2018 @03:16AM (#16357271)
And I constantly get email for other people with my name at gmail.com. Let's see ... I'm married, and I have a few girlfriends, I run marathons, and I've been to several family reunions. Oh, and I'm in the process of buying several different cars. Wheeee!
Re:Let’s talk about usernames by ajdlinux on Monday February 12, 2018 @01:15AM (#16356899)
I suppose the tradeoff here is when you get two non-Gmail users, johndoe and john.doe, who are legitimately different people, one of whom gets very confused and is actually unable to use their correct address.

On the one hand, that's a (presumably extremely) rare corner case - on the other hand, some applications must handle those corner cases.

Re:Let’s talk about usernames by spc476 on Monday February 12, 2018 @03:17AM (#16357277)
Re:Let’s talk about usernames by kbenson on Monday February 12, 2018 @02:48AM (#16357188)
People where trained for that long before Gmail even existed. Email systems have generally followed Postel's law, since it makes life easier for all parties.

Do you want to run a support system where when you ask for people's username or email address, you also have to ask them for their casing (and in the case that their problem is that they didn't know it mattered, they may not know what they signed up for)?

I worked at an ISP for many years. Even though it was all backed by Linux, usernames (which included email addresses) were considered case insensitive for the purpose of the service (all usernames were lowercase). It solves so many problems and the downsides are so small all the sane email providers did it.

The flip side of this is that it was a simpler time. Usernames and email addresses were ASCII, not Unicode. These days with Unicode, you can't even be sure that uppercasing and then lowercasing an already lowercased string yields the same characters.

Re:Let’s talk about usernames by neltnerb on Monday February 12, 2018 @01:51AM (#16357011)
Oh, I assumed the recommendation was to store the actual email address as entered, and then also generate the fingerprint to compare for similarity. Wouldn't you just keep both?
Re:Let’s talk about usernames by mhandley on Monday February 12, 2018 @05:33AM (#16357791)
The problem is that [email protected] and [email protected] can be different people, or [email protected] and [email protected]. On gmail, they can't, but on my university email system for example, this is the default naming convension for circa 50,000 people. If you consider them to be the same, one of them can't sign up to your service. How often this happens? Probably not all that often. But if you run a popular service, it will bite a few people.
Re:Let’s talk about usernames by caf on Monday February 12, 2018 @12:41AM (#16356786)
Yes. The answer to the question posed in the article is actually simple - there are 3 unique email addresses listed. The fact that email addressed to some of them may end up in the same box is neither here nor there.
Re:Let’s talk about usernames by Alex3917 on Monday February 12, 2018 @12:52AM (#16356820)
> The fact that email addressed to some of them may end up in the same box is neither here nor there.

It's not an issue if they're the same, the issue is if they're different. E.g. if I'm storing bitcoin on an exchange (I know) behind the [email protected], then I don't want someone else to be able to register [email protected] and then start looking for bugs in the service or start trying to socially engineer customer support. (The local part of the email address is case sensitive as per the spec.)

Re:Let’s talk about usernames by ubernostrum on Monday February 12, 2018 @01:26AM (#16356943)
You hit the nail on the head.

People can hem and haw about the specs, but at the end of the day Gmail trained most of the world to believe email addresses are case-insensitive, dots don't matter, etc., and now we have to live with the consequences. If that means somebody can't sign up for twelve accounts using case and dot variations of their Gmail, well, so be it. And if that means they come to HN to rant about how that awful site didn't follow the RFCs, then they come to HN to rant about that, but their account will be safer in spite of it.

Re:Let’s talk about usernames by slavik81 on Monday February 12, 2018 @02:00AM (#16357040)
Maybe. But, if I was legitimately [email protected] and you refused to allow me to register an account just because somebody else had a similar email, I'd be pretty annoyed. There's a good chance I'd take my business elsewhere.

I've only once had my email address rejected by a website (for ending with an underscore), but I never bothered setting up another email just for them.

Re:Let’s talk about usernames by ubernostrum on Monday February 12, 2018 @02:08AM (#16357061)
And if the call-center person decided that '[email protected]' should be able to reset the password for '[email protected]' because they read a lifehack article about how dots in email addresses don't matter, you'd take your business elsewhere.

No matter what, there are reasonable hypotheticals where you get angry and take your business elsewhere. The difference is my approach has you leave because you're angry at the signup page, and your approach has you leave because someone stole your stuff. I'll take my approach any day of the week.

Re:Let’s talk about usernames by slavik81 on Monday February 12, 2018 @02:29AM (#16357121)
Given that we're developers and we make the tools that the help desk people use, why would we make a help-desk password reset that can send to a different address than the one registered with the account? Couldn't there be other social engineering tricks people could pull on your help desk employees that you haven't yet imagined? Maybe they also think it's ok to drop the underscore from my email address, despite that not being a part of the gmail scheme.
Re:Let’s talk about usernames by baystep on Monday February 12, 2018 @04:41AM (#16357593)
What about the "I've lost that email account, can I switch it to another to recover my account?" case. Or the "I'm the legal guardian of this person and need the account control switched to my personal email", or in the business world "this employee doesn't work here anymore and as administrator I would like the account transferred to me" cases. Locking customer support systems down tightly has it's own pros/cons.
Re:Let’s talk about usernames by slavik81 on Monday February 12, 2018 @07:28AM (#16358179)
I suppose I was more specific than I really should have been. More broadly, I'm trying to say that you have control over the tools and processes followed by your customer service. They can be used to combat social engineering.

For something as important as the credentials for a bitcoin exchange account, as Alex gave as his example, there should be policies specifying the reasons why account credentials can be changed and what evidence must be presented to do so. Front-line customer service reps shouldn't be flying by the seat of their pants when making difficult decisions with potentially hundreds of thousands of dollars on the line.

Re:Let’s talk about usernames by Alex3917 on Monday February 12, 2018 @09:10AM (#16358624)
What happens when someone calls the CS person and tells them to type in their email address instead of copy pasting it or whatever. If there are any bugs at all in the CS software then it won’t be hard for the CS person to believe there is a bug they need to work around similar to the other bugs that are already in their dashboard.

The point of social engineering attacks is that they’re innocuous requests that don’t raise suspicion, and are hard to train people against.

Re:Let’s talk about usernames by ubernostrum on Monday February 12, 2018 @02:35AM (#16357152)
OK. You go get every single company, site and service on earth to deploy a 100% perfect credential-recovery system and only have it used by 100% perfect people who never ever make a mistake. And when you've finished, let me know and I'll rethink my approach to email.
Re:Let’s talk about usernames by slavik81 on Monday February 12, 2018 @02:50AM (#16357191)
You don't need to make a system that's perfect. You just need to make a system in which checks are required to make _any_ change the credentials on an account. Once those checks are complete, it makes no difference if the change is from [email protected] to [email protected] or to [email protected].
Re:Let’s talk about usernames by Alex3917 on Monday February 12, 2018 @09:04AM (#16358579)
It’s not just password resets that are the issue. It’s also things like:

- Functions to get the account based on the email address

- Internal tools

- Stored procedures and other SQL stuff that happens outside the main code base

- Third-part integrations (Mailgun, Sailthru, ZenDesk, SalesForce, etc.)

That’s a huge attack surface where if there is even a minor mistake by a junior dev that no one noticed then everyone is going to lose their assets under protection.

Re:Let’s talk about usernames by Doctor_Fegg on Monday February 12, 2018 @03:12AM (#16357257)
> Gmail trained most of the world

At least here in the UK, gmail has high reach among techies but is very much a minority provider for normal people. Looking through the 3000ish registered accounts on the local community website I run, Hotmail, Yahoo, and particularly ISPs (btinternet.com, plus.net, sky.com) are all more common than gmail.

Re:Let’s talk about usernames by quickthrower2 on Monday February 12, 2018 @01:16AM (#16356905)
the best way is to err on the side of allowing a single email inbox to create multiple accounts.

Who cares if they do? It's not like that person couldn't create multiple email addresses anyway. E.g. [email protected] and [email protected] can be the same person. You can't validate for that, other than other "gamification" systems (e.g. how HN treats new users - disincentivizes creating multiple accounts because why be limited on your second one when you have a good first one).

by the way I have used the + trick on google to sign up for a service (and pay for it!) that wouldn't let me reuse my old account for some reason. So their relaxed validation made them money.

Re:Let’s talk about usernames by zbentley on Monday February 12, 2018 @09:11AM (#16358625)
It's a difference in difficulty. More to the point, it's a difference in asymmetry of difficulty.

Say you run an online store. You offer USD $5 in first-time user credit.

A lot of people (even some nontechies I know) know about the "+" trick for gmail. Assuming your signup flow is easy and fast, it's very easy for those people to sign up for multiple accounts and get multiple $5 credits.

If a lot of people do that, it might significantly impact your bottom line. Not just because you have to give away a lot of inventory, but secondary effects also suck: you stop the $5-free promotional, and then all of the legitimate users who signed up during the campaign who told their friends to sign up and get some free money now have their friends bad-mouthing the site to them because "it didn't give me free shit the way you told me I should expect it to!". You might see a drop in sign-ups to below pre-promotional levels, or, worse, you might see people who signed up during the promotional trust and use your site less. I know trends/behaviors like this seem trivial--and they definitely would only affect a minority of users--but past a certain scale effects like those can have a real financial impact.

Now let's say your site is "smart" about the "+" trick and doesn't let people with gmail (or google-federated emails--boy, is figuring that out a bastard) accounts sign up multiple times. You'll lose the dubious potential business of folks who like gaming promotionals. You'll still be vulnerable to people creating second email accounts and signing up using those--but the difficulty asymmetry now favors you, the vendor: it's work for a user to make a second email account; work they probably won't do, especially if you blacklist typical temp email services like guerillamail. If the promotional is large enough to entice first-time users but small enough to deter people from doing this, you have succeeded in minimizing your loss. If the user already has a fleet of accounts for this purpose they're probably going to just take your money anyway.

Of course, there's another more annoying scenario which I'll mention because sites should never do it: sites that think they're being smart about the "+" trick by not storing the part of the address between the plus and the domain part. This is usually done to get email campaigns (read: almost entirely spam) to show up in the user's main mailbox rather than some filter-purgatory. It will drive users away in two main ways: first, if I'm technical enough to use the "+" trick and a filter to route mail, I probably have enough obsessive annoyance with spam to immediately either junk-flag yours or delete my account, compared to a small chance I would have actually read it otherwise. Second, more than half of sites that do this which I've audited parse and strip the content of the email address wrongly (parsing emails is very hard, after all) in such a way that what they end up storing could be a totally different person's email, or an invalid one. That means signups just won't work. Whatever you do, don't do this.

This doesn't only apply to first-time-user promotionals, either. It also applies to:

- Referral bonus programs.

- Services that give hand-customized products to users (think a "one per user" etsy store with one person knitting cat dolls or something): multiple similar contacts from the same user would mean you spend a lot of time making their products--time which might be wasted, even if they paid for their products, compared with time spent making them for lots of different users and increasing your recognition/exposure.

- The same applies to tech-support contact-us forms: one user can "bogart" your support staff, clogging the queue with (legitimate or not) requests and defeat your rate limiters by using the "+" trick, making other users wait a long time for replies.

- Others I haven't thought of.

Re:Let’s talk about usernames by lorenzfx on Monday February 12, 2018 @10:36AM (#16359245)
You can still buy your own domain and get as many new email addresses which pass your test as you want. So you probably need another solution to your problem.

What I noticed what several (large) services these days do, is ask for your credit card number on signup (even if they promise not to bill you).

Re:Let’s talk about usernames by zbentley on Tuesday February 13, 2018 @07:03AM (#16366475)
You could. But most people won't. The ones that do this will game your system regardless; they have the time and resources to. It's about making it not worth it to buy a domain and set up MX forwarding/gmail federation or whatever, just for $5.
Re:Let’s talk about usernames by cestith on Monday February 12, 2018 @05:55PM (#16363153)
Many email systems have an idea of a catchall account. I can have a domain for which cestith@mydomain goes into one account and anything else @mydomain goes into a second account.
Re:Let’s talk about usernames by quickthrower2 on Monday February 12, 2018 @04:56PM (#16362690)
Stores that offer $5 credit usually don't sell things that cost $5. So it is nothing but a $5 discount code really.

If they are really giving away credit for free that can be used wholly then they probably need to verify your identity e.g. a $0 credit card authorisation or something.

Re:Let’s talk about usernames by nodesocket on Monday February 12, 2018 @01:17AM (#16356912)
Completely agree. If somebody wants to signup for your service as [email protected] and [email protected] let them. Two distinct user accounts.
Re:Let’s talk about usernames by nzjrs on Monday February 12, 2018 @01:19AM (#16356914)
Dear sir. I enjoyed your free trial. May I have+1 another?
Re:Let’s talk about usernames by Hextinium on Monday February 12, 2018 @01:45AM (#16356987)
If you wanted to continuously have free trials it is trivially easy to just create entirely new email adresses
Re:Let’s talk about usernames by protopeer on Monday February 12, 2018 @07:55PM (#16363861)
yeah, i paid less than a dollar for a super cheap domain, and created a catch-all email forwarder that lets me create any random email forwarding address that all forwards to my one email. i've used this method in the past to get unlimited free things at certain establishments that offer welcome gifts to new rewards accounts.
Re:Let’s talk about usernames by zbentley on Monday February 12, 2018 @09:13AM (#16358643)
I replied to this same idea above: https://news.ycombinator.com/item?id=16358625
Re:Let’s talk about usernames by hungerstrike on Monday February 12, 2018 @01:44AM (#16356985)
Dear user. Sure. Enjoy losing all of your preferences and data every 14 days.
Re:Let’s talk about usernames by tbabb on Monday February 12, 2018 @03:20AM (#16357292)
My thought would be to have two distinct fields: "normalized" and "pristine" version of the user's email. Then always mail to exactly what the user typed ("pristine"), and use the "normalized" mail to prevent multiple registrations to "john.doe@gmail" and "johndoe@gmail" and "johndoe+1@gmail" and "[email protected]" and so on.

Denying registration to a suspected-duplicate seems a lot safer than mailing to a different person.

Re:Let’s talk about usernames by __david__ on Monday February 12, 2018 @05:01AM (#16357686)
If you're going to do that then only do it for email addresses where you understand the local MTA conventions (like gmail). Do not do it for domains you don't know (because you cannot predict how they work).

For instance, on my email domain, [email protected], [email protected], and [email protected] are all the same ('+', '.' and '_' all act like you'd expect '+' to), but [email protected] is not (it get's picked off and aliased somewhere else). Applying gmail conventions to other domains is silly and wrong.

Re:Let’s talk about usernames by sarreph on Monday February 12, 2018 @08:19AM (#16358371)
It does surprise me how many big SAAS products with generous free trials don't check for '+'s on Gmail addresses...
Re:Let’s talk about usernames by pawelk on Monday February 12, 2018 @03:59AM (#16357436)
You can still do a check for the normalized address for known sevices that allow moving the dots and adding a +somthing (e.g. to catch someone creating multiple fake accounts). Just be decent about it and store the user-provided address for communication purposes.
Re:Let’s talk about usernames by Alex3917 on Monday February 12, 2018 @12:18AM (#16356716)
> it's not your identity system's job to deduplicate different logical addresses for the same physical inbox.

The problem is that then you're adding potential attack vectors to every single web app just to cater to the .01% of email clients that insist on implementing the email RFC exactly to spec. Not deduplicating email addresses creates an attack surface for both hard-to-spot technical issues and also social engineering attacks. E.g. what if your ESP at some point adds deduplication on their end, either mistakenly or on purpose, then suddenly you're sending password reset requests to the wrong users.

I think you should normalize email addresses to enforce account uniqueness, both for security purposes and usability, as long as you also store a second copy of the email address exactly as the user entered it and only send email to the latter version.

Re:Let’s talk about usernames by djsumdog on Monday February 12, 2018 @12:48AM (#16356806)
At most you're allowing duplicate accounts to the same e-mail address. If you want to enforce 1 to 1 identities with e-mail accounts, you could have an issue. But in general, it shouldn't be a security concern.