> you don't think engineers who build web crawlers cannot build email classifiers?

We're talking about Google here. It doesn't matter that they have lots of clever people working there; they still occasionally get/guess things wrong, and if you're the unlucky too-small-to-even-notice outfit that happens to get squished by Google today, there's seldom much you can do about it.

Exactly... Outlook by Microsoft is notorious for being very heavy handed with emails, requiring sites to put warnings to users to whitelist their domains so that they receive invoices or notifications.

At this point with outlook it’s pretty much guaranteed that any important “you just paid a ton of money here is the asset you bought” email (show/bus/etc tickets) will go straight to spam. I check spam before I look in the non-“Focused” inbox.

If I send an email to some business, from the Outlook UI, and they reply, Outlook usually classifies the reply as spam. It’s hard to imagine less spammy email than that.

I'd just rename the spam folder to "Inbox-2" or something. ;-)

> Google is pretty accurate in figuring out transactional versus marketing. They don't tell their heuristics, but you don't think engineers who build web crawlers cannot build email classifiers?

Yes, I definitely think that. The engineers can build anything, but where the company focuses matters.

I've seen transactional E-mails get sorted into people's spam/junk/newsletter folders too many times.

I also get tons of spam to my inbox despite regularly marking it as such, so if they are classifying marketing emails, they're not doing anything with that information.

How hard is it to classify a message that literally contains the string "this is an advertisement"?

Your comment literally contains the string "this is an advertisement" but I can't tell what you are advertising?

It's also not an email. I've never seen a legitimate email with that string, and all of the illegitimate ones should be triggering other heuristics, such as the existence of an unsubscribe link, or things like "$x off".

In any case the false positive rate on that would likely be incredibly low, so it's a good heuristic considering how bad the false negative rate is right now.

Dear site admin,

This is as advertisement that appeared on your site yesterday that is a phishing scam pretending to be a bank.

<Screenshot>

Please prevent ads like this showing up on your site.

Regards,

Client XYZ

---

Maybe it's just the positions I've been in, but I've often seen variations of the above email, and I've never seen advertisement emails that flat out say "this is an advertisement"

In fact, what I have seen are advertisement emails of the form

"This is not an advertisement, we'd like to arrange a call to discuss ways to grow your business. Signed, Bob the XYZ product sales manager"

Like others, I get people handing my email out all the time by mistake because I grabbed first-initial-last-name 20 years ago, so I get lots of corporate spam that others have signed me up for. If you look at corporate spam, it frequently contains a passage like this:

> This is an advertisement and outbound email only. Please do not respond to it. This email has been sent on behalf of Kia Motors America, Inc. (KMA). To opt-out of receiving marketing/promotion emails from or on behalf of KMA, please click here.

Or this one I got last week from J Crew:

> We want you to hear about what's just right for you. Update your email preferences here. This email may be considered an advertising or promotional message.

For a while I just ignored it, and this kind of thing never went to spam. Now I always mark it as spam, and it's starting to, but their default spam heuristics are apparently awful, and it seems like marking as spam just affects that one sender, so you have to do it all the time for new spammers. I still just got linkedin spam yesterday after I have marked thousands of their messages. It can't be that hard to come up with heuristics for this. The biggest signal is probably that it contains an unsubscribe link since it has to be there by law.

Your example is also a single message. I imagine they look at patterns, and a single sender sending thousands of emails which are 99% similar is probably also a strong signal that it is spam (yes there are transactional emails that are templated; that's why it's a signal). That combined with the "this is an advertisement" heuristic is probably pretty accurate.

The reality is--obviously--that they are not trying to stop corporate spam. They're an advertising company; they don't want to normalize the idea that advertisements are supposed to be filtered.

Turn off your spam filtering for a week.

You'll find out what the quality of the job they are doing is.

Invoices are in my spam folder regularly. You'd think emails I open consistently month after month, which are followed by receipts would make it through.

Search isn't doing that well either.

Google routinely flags my genuine AWS invoices as possibly dangerous, despite me routinely clicking the "this isn't dangerous, I know what it is" button. So yes, I think it's totally possible that engineers who build web crawlers can't build reliable email classifiers.

Or potentially a low-key sales tactic to push google cloud...?

yeah lol, google is the king of "its not a bug, its a feature".

To be fair to Google, they also flag and filter emails from themselves with some regularity.

And somehow I still get regular “H0ME_DEPOT Order CONFRIMATION” junk landing at the top of my Gmail inbox.

That reminds me: [email protected] is holding my USPS package at the warehouse.

Google seems to do this from ANY mails coming from some of the major VPS providers. They do the same on my linodes as well, despite me having SPF, DKIM, DMARC and even reverse DNS properly configured......

Luckily for me, its mostly just for my own usage and Im not using Google to send anything, its for things like email alerts to my google workspace account...

I dunno, runaway costs on AWS is very dangerous.

"transactional versus marketing"

In my last big job we had big discussions about what is marketing. What can marketing pack into a transactional without it becoming a marketing email? Banner? A tagline in the signature? Testimonials? Also - b/c Germany - big discussions with legal on that topic.

For US companies, the FTC has some guidance on transactional vs marketing, including commingling of the two. https://www.ftc.gov/business-guidance/resources/can-spam-act...

Uh. The answer to this is easy and obvious, unless you are trying to force marketing content into a transaction email.

This is like making technical arguments to someone else that actually, legally, you are not sexually harassing them. If there’s even an inkling of a question, your behavior sucks.

> you don't think engineers who build web crawlers cannot build email classifiers?

Don’t put Google on a pedestal. I’ve seen Google Workspace classify an individual email sent from one colleague to another as spam. Both perfectly legitimate users in the same account / domain. No weird trigger words like Viagra. Just a run-of-the-mill email about work, between two colleagues who had been emailing each other for months. If emails like that aren’t safe from Google’s spam filter, then no emails are safe from Google’s spam filter.

I don’t disagree with you, but before assuming it’s the fault of gmail classifier I would look at Google Workspace admin configuration. There are a lot of settings that admins can tweak and toggle that can mess with email deliverability. You can even create specific rules that only apply when users within the same workspace are emailing each other.

Google Workspace can even be configured to use an external smtp service behind the scenes. Can also be configured to proxy emails through 3rd parties (in which case the email might be leaving the Google ecosystem and then reentering it from a non-Google IP). There’s a lot of silly (seemingly unnecessary) features on the admin side that could trip up a spam filter.

In this case it was a super-basic setup. Nothing weird going on, just an internal email within the same Google account, sent from the Gmail web interface and going straight to the spam folder for the recipient, no filtering rules or anything like that.

Yeah, something wrong with the filter.

Google marked several Samsung mobile phone order confirmation emails as phishing messages a week or two ago. Nobody sells more Android phones than Samsung, so they should be one of Google's top partners to accommodate correctly 100% of the time.

Without knowing anything about the details, is it possible Samsung leaked their SMTP credentials, and Google was seeing phishing to a few of their users?

Yeah that happens all the time, to the point where I had to add specific rules in Google Workspace to never send those to spam. Same with other transactional emails like receipts from some places that I had marked as "not spam" 4 times and Google still couldn't figure out the next one.

> you don't think engineers who build web crawlers cannot build email classifiers?

Nope, I don't. So many things get constantly marked as spam in my inbox, even server notifications, from the same domain, same daily emails, marked repeatedly as "not spam", and added to address book.

Then there's the second problem of google support... your 2fa passwords, email-authentications, password reset links, etc. will be sent out, gmail will send them to spam, your users won't see/find the email, and there's nothing you can do... noone to call at google that would actually listen and try to do anything, no penalties if they don't do anything, only hope that your service is large enough that it gets some traction on twitter or here and some random googler sees it.

You could have put Google in early ‘00s on this pedestal. But the Google today is not worthy of this.

G is like any other Fortune 500 company now. The amount of products in their graveyard grows every year. Maintenance of “legacy” apps is handed off to offshore teams who have objectives to just keep it running until it’s 86’d.

Google has also made plenty of mistakes with web: look at PWAs, AMP, and Chrome just to start.

> you probably should have unsubscribe links

They're not requiring just unsubscribe links. They're specifically requiring "one-click" unsubscribe links that can accept a POST request for unsubscribing. This allows their software to have an unsubscribe button that doesn't require the user to leave their software.

This is the RFC that has to be complied with:

https://datatracker.ietf.org/doc/html/rfc8058

Note, that this is not easy for many people using legacy software. It's a major change. I wouldn't be surprised if this requirement gets delayed multiple times.

Yes, that's the List-Unsubscribe header and it doesn't require a POST request.

Email deliverability has always meant staying on top of changing requirements.

No -- this actually expands on the List-Unsubscribe header, and adds a POST request header for one-click unsubscribe.

From the RFC:

> This document addresses this part of the problem, with an HTTPS POST action

Look at the examples in the RFC for a clear description.

Ah, my mistake. The MAILTO: style unsubscribes were a bit of a pain to deal with anyway.

>this is not easy for many people using legacy software. It's a major change.

That’s fine, I never wanted to receive messages from those people in the first place.

No, that is for generating the Unsubscribe buttons in the email client itself. They also require a link in the body itself. From the google doc:

  Marketing messages and subscribed messages must support one-click unsubscribe, and include a clearly visible unsubscribe link in the message body.

“You need a link not list-unsubscribe” is not fully accurate according to my reading. They are asking senders to support the one click unsubscribe rfc, which uses list-unsubscribe.

> but you don't think engineers who build web crawlers cannot build email classifiers?

I’ve seen Gmail put legit update emails coming from Google itself in spam.

And yet obviously fake Drive shares from "Wells Fargo" or "Chase" get delivered to Inbox