News for hackers. Stuff that matters.
Re:Gmail is opening and caching URLs within emails without u... by whateveracct on Thursday August 19, 2021 @07:05PM (#28240439)
Make the user take action after opening the link. Like click a button.
Re:Gmail is opening and caching URLs within emails without u... by sxp on Thursday August 19, 2021 @07:11PM (#28240501)
And make sure the action is a POST instead of a GET. GETs should never modify important state.
Re:Gmail is opening and caching URLs within emails without u... by sleavey on Friday August 20, 2021 @03:00AM (#28243404)
This is the correct answer. Just because the norm is to embed verification hashes in URLs to be clicked, doesn't mean it's the right way for it to be done.

Why not send a short random code by email for the user to then copy into the sign-up form they were in the process of filling in?

Re:Gmail is opening and caching URLs within emails without u... by vbezhenar on Friday August 20, 2021 @03:28AM (#28243573)
It takes more effort and more users will decide to move elsewhere. I don't really believe that if someone can't bother to copy code from e-mail, he's worthy to have as a client, but some company are obsessed by metrics and percentage of successfully registered users is one of those metrics.
Re:Gmail is opening and caching URLs within emails without u... by toshk on Friday August 20, 2021 @03:37AM (#28243617)
I understand your way of thinking, but we ended up having a flow for a government site where users had 2-3 steps what normally could be done in 1. Also many were not tech savvy and confused. So we ended up adding JS to automate the click.
Re:Gmail is opening and caching URLs within emails without u... by inetknght on Friday August 20, 2021 @12:21PM (#28247808)
You automated a click on a government website? So tell me: how'd that audit go?
Re:Gmail is opening and caching URLs within emails without u... by acdha on Friday August 20, 2021 @10:03AM (#28246091)
This depends on your willingness to turn away business, and may not even be legal depending on where you work. In the United States, I would not want to defend that copy and paste scheme as being compliant with the Americans With Disabilities act having seen usability tests from people trying to accomplish that exact workflow using screen readers. Remember that things like cognitive impairments count and, like vision and motor control/range of motion, most of us will be affected at some point in our lives.

What I do think would be reasonable is having a well-labeled link which takes you to a confirmation form: someone can follow it easily and choose to submit it with far less friction and it leaves standard web semantics intact.

Re:Gmail is opening and caching URLs within emails without u... by duckmysick on Friday August 20, 2021 @03:29AM (#28243575)
Clicking a link (one action) is easier than copying a code and pasting it (two actions). It's possible the user will copy the wrong thing or paste the code into a wrong field, including the browser address bar.

All of that may affect the sign-up rate.

Re:Gmail is opening and caching URLs within emails without u... by ace2358 on Friday August 20, 2021 @04:01AM (#28243764)
Kinda. I often read my email on my phone while working on my desktop. (Or visa versa). In these situations, a code is always better. I hate the links personally.
Re:Gmail is opening and caching URLs within emails without u... by duckmysick on Friday August 20, 2021 @05:58AM (#28244410)
How many times having to click a link (instead of entering a code) stopped you from finishing a sign-up process?
Re:Gmail is opening and caching URLs within emails without u... by prepend on Friday August 20, 2021 @08:44AM (#28245353)
It’s pretty easy to measure. I had a site with a verification step. And we would see like 20% drop off of people who clicked on the link but never confirmed. Not sure why. We didn’t have them copy and paste anything, just click a confirm button.

Switching to no confirm obviously changed this to 0% drop off of people who clicked the link, but the number of people who clicked was the same.

It was curious to me why people wouldn’t go through with the confirmation step, but never learned why. We just learned that for some reason more people click once instead of twice.

Re:Gmail is opening and caching URLs within emails without u... by acdha on Friday August 20, 2021 @10:04AM (#28246107)
How would you have known if that 20% were real people and not bot activity?
Re:Gmail is opening and caching URLs within emails without u... by prepend on Friday August 20, 2021 @02:21PM (#28249549)
I don’t necessarily. But they have active accounts that do stuff and had the drop off activity consistent with “normal users.”

So it doesn’t matter to me if they were bots or not.

For example, 100 users clicked on the first link, 80 completed, and had normal account activity (clicking on stuff, uploading and downloading things, etc).

100 users clicked on the second link and then had normal account activity.

Maybe they were all bots, but they seemed human based on the “normal activity.”

Re:Gmail is opening and caching URLs within emails without u... by tonypace on Friday August 20, 2021 @06:10AM (#28244471)
Nobody remembers the exact moment they stopped thinking about something because it was easier not to.

Ragequitting is one way to exit a process, but just not going to the next step from distraction is surely more common.

Re:Gmail is opening and caching URLs within emails without u... by duckmysick on Friday August 20, 2021 @08:48AM (#28245374)
I'm asking because often when I talk to people about things they hate, they end up admitting it's not that big of a deal. The annoyance is minor enough they don't look for alternatives or abandon whatever they were doing.

The original discussion was about clicking links vs reading and entering the code in sign-up confirmations. The former takes less steps and is easier to complete. Power users with unusual habits might disagree. But if they complete the sign-up anyway, it makes more sense to focus on regular users.

Re:Gmail is opening and caching URLs within emails without u... by prepend on Friday August 20, 2021 @08:41AM (#28245327)
> user to then copy into the sign-up form

Extra steps are hard and boring and people don’t want to do them.

I consider myself a savvy user and I want to click a link. Not click a link, then look up a code from the email, then paste, then click submit.

I’d live with having to manually click “I’m sure I want to unsubscribe” or something.

This is most annoying when the site wants me to type in my email address to unsubscribe. I have lots and lots of different email addresses that funnel into a single one. When the site doesn’t put my address in the “To” field, I dont know who they sent to.

Services should be respectful of users time.

Re:Gmail is opening and caching URLs within emails without u... by 867-5309 on Friday August 20, 2021 @10:17AM (#28246272)
>This is most annoying when the site wants me to type in my email address to unsubscribe.

I've become increasingly suspicious of this practice

if my email address is in the URL, why don't you autofill that email box for me? if it's not in the URL, why aren't you fetching it from your database using my unique hash in the URL? do you even keep any records of email subscription preferences? am I just signing up for more shitty spam by giving you my email, again? am I just being marked as 'active' i.e. fresh meat, somewhere in the spammiverse?

these questions become more poignant and the suspicion more fiery when, low and behold, it turns out you are still subscribed

I don't fill them in any more, I just block the sender

don't get me started on the "it may take up to 28 days for our systems to register your desertion" bollocks -- I'm not working a contractual notice period, or running a lap of dishonour. it's a bitflip to 'false' in the 'is_pesterable' column, else a respectful deletion. it takes microseconds, not weeks!

Re:Gmail is opening and caching URLs within emails without u... by prepend on Friday August 20, 2021 @02:22PM (#28249568)
That too as I suspect the unsubscribe links may just be data gathering.

I typically never type in any information because I assume the site doesn’t know and wants to know.

I’d rather just set up a kill rule on my end than risk getting my email on one more list.

Re:Gmail is opening and caching URLs within emails without u... by nickjj on Friday August 20, 2021 @09:43AM (#28245831)
There were good suggestions in other comments in this HN post.

One of them mentioned that you can continue keeping things as a 1 click solution with the token in the URL, but instead of doing the destructive action upon visiting the link -- instead you would get sent to a page with a form where the token is put into a hidden field that gets auto-submit as a POST request with Javascript.

This way from your POV it's a 1 click solution. You only waste a second waiting for the redirect and if the user doesn't have Javascript enabled you can <noscript> the field as being an input field which is pre-filled out based on the value from the URL (this can be done server side).

Now everyone is happy, unless gmail is going to go as far as auto-following redirects with JS enabled.

Re:Gmail is opening and caching URLs within emails without u... by wutbrodo on Friday August 20, 2021 @09:56AM (#28246000)
Ideally they wouldn't be following redirects _if they're POST requests_, right?
Re:Gmail is opening and caching URLs within emails without u... by nickjj on Friday August 20, 2021 @10:06AM (#28246128)
True, that was a bad choice of words but gmail could still load the page, execute the JS which in turn submits the form which is a POST request to your back-end. It's technically not following a redirect but it's doing things beyond just visiting the URL linked in the email due to it executing JS.
Re:Gmail is opening and caching URLs within emails without u... by wutbrodo on Friday August 20, 2021 @12:35PM (#28248025)
Ah interesting, so you're saying that GMail is not likely to be avoiding the POST requests if they're in the JS code? I have a passing, non-professional familiarity with Web practices, so this isn't something I have a great intuition for.
Re:Gmail is opening and caching URLs within emails without u... by nickjj on Friday August 20, 2021 @02:02PM (#28249304)
The basic flow would be:

    - You GET /reset/abc123
    - Your server responds back with a page that has a form
    - There's a hidden field with the token
    - Javascript kicks in and on page load executes the form as a POST request
    - Your server responds to that POST request and does whatever it needs to do
All of that is kicked off by gmail visiting /reset/abc123, and now it comes down to whether or not gmail's pre-visiting code will run the JS on the page. If not, then the above workflow fixes this issue, if it does then you're in the same position as avoiding all of this and having a GET /reset/abc123 perform the destructive action.
Re:Gmail is opening and caching URLs within emails without u... by wutbrodo on Friday August 20, 2021 @02:30PM (#28249663)
Right, it's the second part of the fourth bullet that I was asking about. Basically, that it sounds like it isn't feasible for Google to avoid POST requests if they're being submitted from JS.
Re:Gmail is opening and caching URLs within emails without u... by sleavey on Friday August 20, 2021 @09:06AM (#28245500)
Are we really going to continue to break the paradigm that GET requests should be idempotent to save people an extra click or Ctrl+C and Ctrl+V? Standards matter. In this case Google are doing something that should be allowed, but being criticised for it because it breaks badly implemented services.

Entering emailed or texted codes is becoming more common with 2FA for banking, PayPal etc. anyway so I think most people are going to broadly manage.

Re:Gmail is opening and caching URLs within emails without u... by prepend on Friday August 20, 2021 @02:27PM (#28249624)
Sorry, GET requests aren’t idempotent. At the minimum they create log entries. So you can DDoS servers by filling their logs with “idempotent” GETs.

UX is important, and I think saying “suck it users, I’m going to use GET the way I think is write” is not a positive way of thinking about it.

I think the problem is just the mechanics of POST not being allowed in an email, so if there’s a way to POST from just clicking on a link I think we should use it. But there’s not, so having a GET that triggers something is the least bad thing. I like it better than javascript and forms in email. And better than autosubmitting, hidden forms on load.

Re:Gmail is opening and caching URLs within emails without u... by Cthulhu_ on Friday August 20, 2021 @07:35AM (#28244902)
Verification hashes in URLs are fine, as long as accessing the URL does not invalidate the hash yet.
Re:Gmail is opening and caching URLs within emails without u... by UI_at_80x24 on Friday August 20, 2021 @08:11AM (#28245144)
This is how Steam does it.
Re:Gmail is opening and caching URLs within emails without u... by _wldu on Thursday August 19, 2021 @07:09PM (#28240477)
Thanks. That's good advice.
Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Friday August 20, 2021 @02:22AM (#28243218)
Btw you could just have JS do a POST request, the user doesn't need to do anything except open the page. This is how unsubscribe pages work.
Re:Gmail is opening and caching URLs within emails without u... by _wldu on Friday August 20, 2021 @07:14AM (#28244783)
Thanks. I don't use JS, just Go with HTML templates. I populate the form now with {{ .code }} from the URL so the user does not have to copy/paste the code. But they do have to click 'Submit' to post the form. I think this is a reasonable approach that most users are OK with.
Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Friday August 20, 2021 @08:54AM (#28245416)
You could add a single line of JS to have it auto submitted as well
Re:Gmail is opening and caching URLs within emails without u... by _wldu on Friday August 20, 2021 @09:18AM (#28245585)
Wouldn't that be equivalent to just doing a GET?
Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Friday August 20, 2021 @11:05AM (#28246866)
No... Because you have to do a GET, execute JavaScript, and make a POST request. Bots don't execute JavaScript and no well-behaved bots are going to make POST requests
Re:Gmail is opening and caching URLs within emails without u... by SamBam on Friday August 20, 2021 @09:30AM (#28245703)
No, because if you curl the original url the POST for the second url won't be triggered, but if you navigate your browser to the url, the browser will trigger the POST.
Re:Gmail is opening and caching URLs within emails without u... by kevin_thibedeau on Friday August 20, 2021 @03:02AM (#28243418)
That presumes everyone executes random JS or has a browser that supports it.
Re:Gmail is opening and caching URLs within emails without u... by input_sh on Friday August 20, 2021 @03:44AM (#28243651)
Create a "click here" button, hide it by adding it a class or a style with JS.
Re:Gmail is opening and caching URLs within emails without u... by kevin_thibedeau on Saturday August 21, 2021 @12:04AM (#28254075)
There is this thing called an HTML form. Every browser supports it.
Re:Gmail is opening and caching URLs within emails without u... by aendruk on Friday August 20, 2021 @03:10AM (#28243466)
It should go without saying that you provide a fallback.
Re:Gmail is opening and caching URLs within emails without u... by bawolff on Friday August 20, 2021 @06:07AM (#28244460)
Depending on context and implementation details, this can often be a security issue (csrf or something similar). Probably not in the unsubscribe case though
Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Friday August 20, 2021 @11:03AM (#28246839)
Obviously you need to make sure your API is not susceptible to CSRF but that goes without saying... Should I also tell him to password protect his database? :P
Re:Gmail is opening and caching URLs within emails without u... by bawolff on Friday August 20, 2021 @12:12PM (#28247689)
If you follow what the parent is suggesting (open a page with a get request that has js which does a post request automatically with no user interaction), its probably impossible to not be susceptible to csrf
Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Friday August 20, 2021 @08:34PM (#28252957)
I'm really not sure why you think that, but that's just not true at all. For starters, if it is a JSON API then a CORS request will be done for all cross-domain API requests. Even a missing CORS configuration would then block the CORS request from third-party domains.
Re:Gmail is opening and caching URLs within emails without u... by bawolff on Saturday August 21, 2021 @01:13AM (#28254404)
So the situation is: there is some url you open (with a normal get request. Typically but not neccesarily from an email), then that url does the non-idempotent POST request without any futher user interaction.

A malicious page that knows the url used in the email could open the url from the email in a popup. The js will execute in the popup, and do the POST request. It doesn't matter how much csrf protection you have on the POST step, if anyone can trigger it with no user interaction just by opening some page with a GET request.

Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Saturday August 21, 2021 @01:40AM (#28254522)
> A malicious page that knows the url used in the email could open the url from the email in a popup

Sorry but this whole scenario is just ridiculous. If somebody can access your email it is already game over. It doesn't matter what web technology you are using at the that point, user interaction or not.

If a "malicious page" knows the URL it doesn't matter at all because it means it is capable of arbitrary code execution and that point it could just exfiltrate the URL to somebody or a Chromium instance to perform the user interaction. Actually if the page can open a popup I think it could also execute JavaScript within the context of the page and perform the user interaction right there.

Re:Gmail is opening and caching URLs within emails without u... by bawolff on Saturday August 21, 2021 @03:39AM (#28255032)
So there's two scenarios:

Scenario a) No authentication-y bits in the url. User goes to the url, site checks if the user is already logged in via a cookie. If so, does the POST request.

Typically in this case the urls are easily guessable, so that's an easy CSRF. In principle they could be made per-user (some sort of HMAC on a user+timestamp). In practise, I think its fairly common for websites not to do that in this sort of situation.

scenario b) The url contains some sort of nonce, or signed assertion that automatically logs the user in. I think this is fairly common in email urls, because web developers want people to be able to take an action just from clicking the link in the email, even if the device they read email on is not the same as the device they normally use to interact with the application. I also think this is the scenario that applies to this discussion, since it was started around talking about the issues caused by google auto following links, and in scenario A, google auto-following links would not be an issue.

Of course, in principle, its possible that the authentication bits in the url, just authenticate that action, and don't generally log the user in. In practise I think its really common to just generally log the user in, since most sites want people to stay on the site once the user does anything, and not just immediately exit after the email action is completed.

These urls are typically not guessable. A small percentage of users do tend to smatter these across the internet (e.g. https://urlscan.io/), but ignoring that, this is a login-CSRF. That is, an attacker can generate their own such url, and force their victim to log into the attacker's account.

The impact of a login-csrf tend to be very application specific. Sometimes its kind of minor, but I have definitely seen cases in major websites where a login-csrf can lead to a full account take-over of the victim's account.

> Actually if the page can open a popup I think it could also execute JavaScript within the context of the page and perform the user interaction right there.

This is only true if the pop-up has the same origin as the site that opened it. Otherwise there is just a very limited API (Basically, postMessage(). Also both sides can change the current url of the other side, which is a bit nuts). Also there is now a new http header, Cross-Origin-Opener-Policy that affects this.

Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Saturday August 21, 2021 @03:55AM (#28255086)
What you are describing isn't even CSRF
Re:Gmail is opening and caching URLs within emails without u... by debaserab2 on Saturday August 21, 2021 @09:59PM (#28262236)
Yes it is - and it's worthwhile to read bawolffs well written explanation of how exactly it could be exploited. Downplaying security vulnerabilities of this sort is precisely how database leaks happen.
Re:Gmail is opening and caching URLs within emails without u... by weird-eye-issue on Tuesday August 24, 2021 @03:25AM (#28285970)
I don't downplay security issues. I just make sure they are actually understood first and that isn't what is going on here at all. What he is describing is not related to CSRF
Re:Gmail is opening and caching URLs within emails without u... by bradknowles on Monday August 23, 2021 @03:02AM (#28273013)
Make the link password protected, and they don’t get the password in the same e-mail message.

Kinda hard to pre-scan a URL if you can’t provide the password for it.