Their algorithms very likely look at (I hope so, at least) spam marking rates. I would bet that users mark promotional emails at a order of magnitude higher rate than transactional emails.

You're talking about Transactional emails? You cant unsubscribe from TRANSACTIONAL emails. That's why they're transactional...not marketing. It's really important to differentiate that.

I "know" that.

I'm asking how does Google differentiate between a transactional and a non transactional email?

They also say in their guidelines

> *Marketing messages and subscribed messages* must support one-click unsubscribe, and include a clearly visible unsubscribe link in the message body.

So how is Google determining what is a Marketing/Subscribed message? If they're not, then am I required to tack on this header to ALL emails regardless of type or risk getting binned?

Ahhh I see what you mean now, but it wasn't clear in your initial question.

Gmail's algorithms analyzes, and has been doing over the last ~20 years, a combination of factors to classify emails as promotional or transactional!

Nothing in the code itself of your email will indicate that, other than the presence of an unsub link + the rest of the footer (which is the obvious sign that's a marketing email)

If you’re sending transactional emails like password resets or MFA, then the emails will have close to a 100% open rate. This is (likely) an important factor that Google uses to judge whether email is transactional, or more generally whether it is desired by recipients, alongside other factors like having a very low complaint rate.

100% open rate on transactional emails feels too high to me. Something like an e-commerce purchase might kick off multiple emails (purchase made, shipped, arrived), none of which the user opens

Sorry, to clarify, I only mean this particular type of transactional email: password reset, MFA.

But even for other types of transactional emails, like shipment confirmations, I would expect the open rate to be much higher and/or the complaint rate to be much lower than for marketing email.

It’s also not a bad idea to provide an unsubscribe option for shipment updates.

Kicking off a chain of emails a user cannot easily opt out of could well be the sort of emails users want to lose. There probably should be a one-click 'stop emailing me' button, for this and future purchases. Which would be a support burden, yes.

I watch for the subject line. I don't actually care what the content says...

So... let's assume many users do this, and let's assume Google factors in the opening rate into the transactional-email-likeness score, and that transactional-email-senders become widely aware of this...

Then senders' incentive will become to make the subject line into clickbait for the content, so that you'll open the message. So instead of subjects like "Order placed", "Order paid", "Order shipped", "Order out for delivery" you'll get uniform subjects along the lines of "IMPORTANT UPDATE TO YOUR ORDER". You will lose efficiency getting through your emails, and over time the metric will lose its indicativeness. Everybody loses.

Some of these emails are legally required for online shops. Doesn't matter if the user wants to receive them or not, they _have to_ be sent and actually delivered to the user's inbox.

I'm not sure how the 'actually delivered' would be enforced. Does Google have an affirmative requirement to deliver a 3rd parties message? I hope not.

My gmail address received 35 emails yesterday (which didn't get spam filtered). All but 3 of those got auto-archived by the filters I have in gmail. I would love google to just do this automatically.

Practically I might need another message or two a week that didn't hit my inbox.... but that's fine as long as it's as it is still searchable.

We’ve received your order … we’ve taken payment for your order … your order has left our warehouse … your order has arrived in another warehouse … your order is with a delivery driver … all for a $5 cable.

> If you’re sending transactional emails like password resets or MFA, then the emails will have close to a 100% open rate.

So I can disable a competitor’s email functionality by triggering a whole bunch of password reset requests for all discoverable usernames?

That could potentially cause them problems, yeah, if you were able to do that endlessly. In practice most companies will have some kind of rate limiting in place around features like that (by IP, cookie, captcha, etc.)

IP and cookie-based rate-limiting are trivially bypassed. In fact, any kind of rate-limiting is ineffective here, especially for smaller organisations, because you only need to generate a small fraction of the traffic they normally send out. If they separate transactional mail from other types of mail (something that is frequently recommended), then how many illegitimate password reset emails do you think an attacker needs to trigger to get to, say, a 5% failure rate? Smaller organisations don’t send out an awful lot of transactional email.

True, but modern CAPTCHA rate limiting is not easily bypassed, and a lot of the solutions are free.

Together with cookies, you can show the captcha only to visitors that are not already recognized in some way, giving them a limited number of actions before showing the captcha. And regardless of whether you want one on your password reset page, you almost certainly want one on your login page anyway.

If they support SMS 2FA they need to be prepared for this too because it costs a lot. Yeah, so people need to ensure that reset is at least a little hard to abuse. After all, it's a bad experience for their users if they receive a shit ton of reset emails anyway.

Can confirm that last part. I get so many Facebook reset messages, it's a bit ridiculous.

I rarely open 2FA emails, because usually the displayed preview is all I need.

I open way less than 100% of password resets - because some are malicious.

Well, I wish my ISP would stop marking ads and promotions as "transactional". Just because they have a system that prohibits unsubscribing, doesn't mean they should be allowed to abuse that system.

I've had a website that sent me no fewer than 6 emails over the course of 10 days for a single transaction, 5 of which were full of ads and links to their website and products. I emailed them and asked them to stop and their response was there was no way to opt out, they were transactional for new accounts.

My ISP constantly sends me emails about "staying safe online" and "the holiday season". At the bottom of the email, it says "THIS IS A SERVICE-RELATED EMAIL", supposedly to excuse the lack of any unsubscribe link.

Unfortunately you are no longer allowed to take them to court over this, as their terms of service simply say you are no longer allowed to sue them :) just like all tech companies that know they're committing lawsuit-worthy offenses.

Maybe transactional emails don't need an unsubscribe link like marketing emails, but they do need a "not my account; please stop" link to avoid the spam button.

Why would you be receiving transactional emails for an account that isn't yours?

Hah. I have josephg@gmail. I sometimes wake up to threads of 6+ password reset attempts over an hour from someone who doesn’t know their own email address. For a couple years I got pay stubs. And monthly cell phone invoices from India.

I think that email address gets more email for other people than email for me at this point.

Same. Flights trips -- including PNR. Invoices. School reports. Tons of telephone bills. Frequent Uber trips (somewhere in Africa, for some reason). The list goes on and on. And my email address is short but not that common, but still get hit a few times per week.

It really drives me crazy that none of them have any type of email confirmation before accepting an email address as valid.

Because many people are not great at entering their email addresses correctly and many sites don't require any sort of address verification/confirmation.

If you have a common word or common name email address at a big email provider then you almost certainly are getting: password reset emails, billing invoices/order confirmations, tax info, childcare/education notices, medical appointment confirmations, local government notices, business conversations, wedding invitations, etc.

All legitimate and not spam but intended for a different recipient.

Someone signing up with a wrong email

Another source that are common are emails entered in on physical point of sale devices

my wife gets these regularly. there's a few people in the UK (we're in the US) that have similar gmail addresses as hers, and use her email address often. she'll get restaurant reservation notices, dr appt confirmations, tv repair schedule confirmations, delivery notices, etc. She's called the vendors a couple of times, and also called the people directly a a couple times. "You've entered your email wrong, please stop using my email".

One person, one time, understood the situation, thanked her, and updated things. And a year later, we got email for them. There's lots of mischief we could get up to, if so inclined, but we're not like that.

Someone last year accused her of 'hacking' in to their computer and stealing emails, so she's basically given up. But these people are missing their dr appointments, delivery change notifications, etc. And by 'these people', I'm meaning - it's perhaps 4 other people with slight variations of the same spelling.

This is a frequent occurrence for anyone with a common name.

https://xkcd.com/1279

Lack of opt-in into those will have me keep marking those as spam. Just like those US political newsletters that also don’t feel like they need to verify mails.

US political emails are even more annoying when you aren’t American. I flag all that stuff as spam without hesitation. If you do that, I hope your entire domain ends up flagged as spam.

I had the idea to do a 1 dollar donation and then see the campaign getting flagged for illegal campaign contributions, but that is probably illegal for me as well.

(non-us based not us citizen)

use different subdomains for transactional and marketing emails.

Its 5000/day for marketing, and if you are sending 5000 emails a day, you probably should have unsubscribe links. https://support.google.com/mail/answer/81126#requirements-5k You also need a link, not just list-unsubscribe, and it is specifically for marketing emails.

In my experience, Google is pretty accurate in figuring out transactional versus marketing. They don't tell their heuristics, but you don't think engineers who build web crawlers cannot build email classifiers? They have reliably been sorting my promotional emails from transaction emails for almost a decade now.

But off the top of my head when working on an email marketing platform: sender address, message subject and content, single message or bulk inbound at a given time, open rates, click rates, unsub rates, bounce rates. Part of sender reputation is ESPs building a profile of what kind of email you send from an address.

> you don't think engineers who build web crawlers cannot build email classifiers?

We're talking about Google here. It doesn't matter that they have lots of clever people working there; they still occasionally get/guess things wrong, and if you're the unlucky too-small-to-even-notice outfit that happens to get squished by Google today, there's seldom much you can do about it.

Exactly... Outlook by Microsoft is notorious for being very heavy handed with emails, requiring sites to put warnings to users to whitelist their domains so that they receive invoices or notifications.

At this point with outlook it’s pretty much guaranteed that any important “you just paid a ton of money here is the asset you bought” email (show/bus/etc tickets) will go straight to spam. I check spam before I look in the non-“Focused” inbox.

If I send an email to some business, from the Outlook UI, and they reply, Outlook usually classifies the reply as spam. It’s hard to imagine less spammy email than that.

I'd just rename the spam folder to "Inbox-2" or something. ;-)

> Google is pretty accurate in figuring out transactional versus marketing. They don't tell their heuristics, but you don't think engineers who build web crawlers cannot build email classifiers?

Yes, I definitely think that. The engineers can build anything, but where the company focuses matters.

I've seen transactional E-mails get sorted into people's spam/junk/newsletter folders too many times.

I also get tons of spam to my inbox despite regularly marking it as such, so if they are classifying marketing emails, they're not doing anything with that information.

How hard is it to classify a message that literally contains the string "this is an advertisement"?

Your comment literally contains the string "this is an advertisement" but I can't tell what you are advertising?

It's also not an email. I've never seen a legitimate email with that string, and all of the illegitimate ones should be triggering other heuristics, such as the existence of an unsubscribe link, or things like "$x off".

In any case the false positive rate on that would likely be incredibly low, so it's a good heuristic considering how bad the false negative rate is right now.

Dear site admin,

This is as advertisement that appeared on your site yesterday that is a phishing scam pretending to be a bank.

<Screenshot>

Please prevent ads like this showing up on your site.

Regards,

Client XYZ

---

Maybe it's just the positions I've been in, but I've often seen variations of the above email, and I've never seen advertisement emails that flat out say "this is an advertisement"

In fact, what I have seen are advertisement emails of the form

"This is not an advertisement, we'd like to arrange a call to discuss ways to grow your business. Signed, Bob the XYZ product sales manager"

Like others, I get people handing my email out all the time by mistake because I grabbed first-initial-last-name 20 years ago, so I get lots of corporate spam that others have signed me up for. If you look at corporate spam, it frequently contains a passage like this:

> This is an advertisement and outbound email only. Please do not respond to it. This email has been sent on behalf of Kia Motors America, Inc. (KMA). To opt-out of receiving marketing/promotion emails from or on behalf of KMA, please click here.

Or this one I got last week from J Crew:

> We want you to hear about what's just right for you. Update your email preferences here. This email may be considered an advertising or promotional message.

For a while I just ignored it, and this kind of thing never went to spam. Now I always mark it as spam, and it's starting to, but their default spam heuristics are apparently awful, and it seems like marking as spam just affects that one sender, so you have to do it all the time for new spammers. I still just got linkedin spam yesterday after I have marked thousands of their messages. It can't be that hard to come up with heuristics for this. The biggest signal is probably that it contains an unsubscribe link since it has to be there by law.

Your example is also a single message. I imagine they look at patterns, and a single sender sending thousands of emails which are 99% similar is probably also a strong signal that it is spam (yes there are transactional emails that are templated; that's why it's a signal). That combined with the "this is an advertisement" heuristic is probably pretty accurate.

The reality is--obviously--that they are not trying to stop corporate spam. They're an advertising company; they don't want to normalize the idea that advertisements are supposed to be filtered.

Turn off your spam filtering for a week.

You'll find out what the quality of the job they are doing is.

Invoices are in my spam folder regularly. You'd think emails I open consistently month after month, which are followed by receipts would make it through.

Search isn't doing that well either.

Google routinely flags my genuine AWS invoices as possibly dangerous, despite me routinely clicking the "this isn't dangerous, I know what it is" button. So yes, I think it's totally possible that engineers who build web crawlers can't build reliable email classifiers.

Or potentially a low-key sales tactic to push google cloud...?

yeah lol, google is the king of "its not a bug, its a feature".

To be fair to Google, they also flag and filter emails from themselves with some regularity.

And somehow I still get regular “H0ME_DEPOT Order CONFRIMATION” junk landing at the top of my Gmail inbox.

That reminds me: [email protected] is holding my USPS package at the warehouse.

Google seems to do this from ANY mails coming from some of the major VPS providers. They do the same on my linodes as well, despite me having SPF, DKIM, DMARC and even reverse DNS properly configured......

Luckily for me, its mostly just for my own usage and Im not using Google to send anything, its for things like email alerts to my google workspace account...

I dunno, runaway costs on AWS is very dangerous.

"transactional versus marketing"

In my last big job we had big discussions about what is marketing. What can marketing pack into a transactional without it becoming a marketing email? Banner? A tagline in the signature? Testimonials? Also - b/c Germany - big discussions with legal on that topic.

For US companies, the FTC has some guidance on transactional vs marketing, including commingling of the two. https://www.ftc.gov/business-guidance/resources/can-spam-act...

Uh. The answer to this is easy and obvious, unless you are trying to force marketing content into a transaction email.

This is like making technical arguments to someone else that actually, legally, you are not sexually harassing them. If there’s even an inkling of a question, your behavior sucks.

> you don't think engineers who build web crawlers cannot build email classifiers?

Don’t put Google on a pedestal. I’ve seen Google Workspace classify an individual email sent from one colleague to another as spam. Both perfectly legitimate users in the same account / domain. No weird trigger words like Viagra. Just a run-of-the-mill email about work, between two colleagues who had been emailing each other for months. If emails like that aren’t safe from Google’s spam filter, then no emails are safe from Google’s spam filter.

I don’t disagree with you, but before assuming it’s the fault of gmail classifier I would look at Google Workspace admin configuration. There are a lot of settings that admins can tweak and toggle that can mess with email deliverability. You can even create specific rules that only apply when users within the same workspace are emailing each other.

Google Workspace can even be configured to use an external smtp service behind the scenes. Can also be configured to proxy emails through 3rd parties (in which case the email might be leaving the Google ecosystem and then reentering it from a non-Google IP). There’s a lot of silly (seemingly unnecessary) features on the admin side that could trip up a spam filter.

In this case it was a super-basic setup. Nothing weird going on, just an internal email within the same Google account, sent from the Gmail web interface and going straight to the spam folder for the recipient, no filtering rules or anything like that.

Yeah, something wrong with the filter.

Google marked several Samsung mobile phone order confirmation emails as phishing messages a week or two ago. Nobody sells more Android phones than Samsung, so they should be one of Google's top partners to accommodate correctly 100% of the time.

Without knowing anything about the details, is it possible Samsung leaked their SMTP credentials, and Google was seeing phishing to a few of their users?

Yeah that happens all the time, to the point where I had to add specific rules in Google Workspace to never send those to spam. Same with other transactional emails like receipts from some places that I had marked as "not spam" 4 times and Google still couldn't figure out the next one.

> you don't think engineers who build web crawlers cannot build email classifiers?

Nope, I don't. So many things get constantly marked as spam in my inbox, even server notifications, from the same domain, same daily emails, marked repeatedly as "not spam", and added to address book.

Then there's the second problem of google support... your 2fa passwords, email-authentications, password reset links, etc. will be sent out, gmail will send them to spam, your users won't see/find the email, and there's nothing you can do... noone to call at google that would actually listen and try to do anything, no penalties if they don't do anything, only hope that your service is large enough that it gets some traction on twitter or here and some random googler sees it.

You could have put Google in early ‘00s on this pedestal. But the Google today is not worthy of this.

G is like any other Fortune 500 company now. The amount of products in their graveyard grows every year. Maintenance of “legacy” apps is handed off to offshore teams who have objectives to just keep it running until it’s 86’d.

Google has also made plenty of mistakes with web: look at PWAs, AMP, and Chrome just to start.

> you probably should have unsubscribe links

They're not requiring just unsubscribe links. They're specifically requiring "one-click" unsubscribe links that can accept a POST request for unsubscribing. This allows their software to have an unsubscribe button that doesn't require the user to leave their software.

This is the RFC that has to be complied with:

https://datatracker.ietf.org/doc/html/rfc8058

Note, that this is not easy for many people using legacy software. It's a major change. I wouldn't be surprised if this requirement gets delayed multiple times.

Yes, that's the List-Unsubscribe header and it doesn't require a POST request.

Email deliverability has always meant staying on top of changing requirements.

No -- this actually expands on the List-Unsubscribe header, and adds a POST request header for one-click unsubscribe.

From the RFC:

> This document addresses this part of the problem, with an HTTPS POST action

Look at the examples in the RFC for a clear description.

Ah, my mistake. The MAILTO: style unsubscribes were a bit of a pain to deal with anyway.

>this is not easy for many people using legacy software. It's a major change.

That’s fine, I never wanted to receive messages from those people in the first place.

No, that is for generating the Unsubscribe buttons in the email client itself. They also require a link in the body itself. From the google doc:

  Marketing messages and subscribed messages must support one-click unsubscribe, and include a clearly visible unsubscribe link in the message body.

“You need a link not list-unsubscribe” is not fully accurate according to my reading. They are asking senders to support the one click unsubscribe rfc, which uses list-unsubscribe.

> but you don't think engineers who build web crawlers cannot build email classifiers?

I’ve seen Gmail put legit update emails coming from Google itself in spam.

And yet obviously fake Drive shares from "Wells Fargo" or "Chase" get delivered to Inbox

Also forcing people to click on opaque links in random emails cannot end good.

Well the answer of course is for google to clone the unique features of your service and classify your site and its outgoing emails as spam.

Our (Postmark's) take is that while these changes are explicitly for marketing/bulk email senders, they are coming for transactional emails as well sooner or later: https://postmarkapp.com/blog/2024-gmail-yahoo-email-requirem...