News for hackers. Stuff that matters.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by stefan_ on Thursday January 30, 2020 @02:46PM (#22194723)
So under the guise of spam protection, we have created an oligopoly of email gateways, despite sending an email being an utterly trivial, long solved problem, and now those companies are putting the squeeze on.

The best part about this particular item on the list of "things that have only gotten worse over time": they charge companies more to deliver what is essentially spam, and they conspired to make that spam show up in inboxes and "enhance" it with all sorts of tracking.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by Eikon on Thursday January 30, 2020 @02:59PM (#22194867)
> despite sending an email being an utterly trivial, long solved problem.

It is absolutely not. Even if you are not considering the whole spam thing.

Sending an email with the multitude of clients, MIME which is very complex in itself, network issues, message queuing, retries, unsubscription, bounces, providers feedback loops, rate limiting and what not is very hard.

If you add the whole spam folder + blacklists situation into consideration, sending emails becomes a more than painful thing you don't want to deal with.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by kick on Thursday January 30, 2020 @03:29PM (#22195192)
Sending an email is absolutely a solved problem.

I can send a message to my address in 2020 using a 2003 Fedora Core 1 box without a domain using only what's part of the POSIX standard, and I'll receive it just fine.

I just did a few days ago, actually. It even got past my provider's spam filter.

I could probably have done it using earlier software than 2003-era mail/mailx, but that was the quickest way I could find to send an attachment given the software on hand.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ebiester on Thursday January 30, 2020 @03:34PM (#22195246)
You can still do that.

However, I, as a receiver of email, get thousands of spam messages a day. So, I use services that block them before they even reach my view, or get sent directly to the spam folder.

If you want to send me an email, that's solved. If you want to send it to ten thousand people on your mailing list, and not have it end up in the spam folder, you may have more work.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by staticautomatic on Thursday January 30, 2020 @03:44PM (#22195340)
This raises what's really the ultimate question, which is whether and how much individuals should be made to suffer so that advertisers can send spam. Certainly the recipients are not responsible for the advertisers' decisions to spam, or more charitably, to blur the lines. Yet it's individuals who pay, in this case literally, for the advertisers' privilege of spamming them. It's extremely fucked.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by edmundsauto on Thursday January 30, 2020 @06:00PM (#22196898)
I think this is what happens in open systems. The alternative is walled gardens, like FB messages. Even those suffer from spam, although it's much less of the wild west.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mc3 on Thursday January 30, 2020 @09:47PM (#22198530)
The alternative is simple:

For stranger to stranger communication, both business and non-business set up a web form. With a challenge if needed.

For friend to friend communication, use email, friends' address in on a white list.

For business to consumer communication, white list is again used.

For spammer to anyone communication, including business lists that refuse to take you off, not on white list so doesn't get in. Bounce response with reason "non on white list"

It's a cultural change though: an email address won't get through unless you are unblocked, but technically easy.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by sokoloff on Thursday January 30, 2020 @10:12PM (#22198669)
What's the easy/low-friction way for Joe Random Consumer to manage their whitelists?

An email comes in from a sender address which isn't on the whitelist, but is (in fact) a friend. What's the handling?

An email comes in from a sender address which isn't on the whitelist, but is not (in fact) a friend. What's the handling?

Do I never hear from the first friend? Or do I get bombarded with "hey, read this email and see if the sender is a friend" a bunch of times?

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ezrast on Friday January 31, 2020 @03:26AM (#22200024)
I'm just shooting from the hip here, but I think it's:

Joe manages his whitelist through his mail provider's web UI. Many providers already use the address book as a whitelist; the only difference is they default to filtration instead of assuming spam.

Joe's friend's mail gets routed to the spam box. Joe's friend gets a bounce notification that says "To be added to Joe's whitelist, click here (and optionally solve a captcha/enter Joe's dog's name/submit a blood sample/deposit $0.25 worth of bitcoin into Joe's wallet)." Joe's friend clicks here, and Joe's mail provider adds Joe's friend to the whitelist and promotes the email to the inbox.

Joe's spammer gets the same message and disregards it. Their email stays in spam forever.

Joe's spam box gets bombarded with "hey, read this email and see if the sender is a friend" a bunch of times. He can trawl through them for actual friends if he wishes.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mc3 on Thursday January 30, 2020 @10:23PM (#22198732)
Fair game. You got me, I have no nice answer other than phone your friend. I'll stop digging. I'm not going to do what this guy did: https://news.ycombinator.com/item?id=16862739
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by edmundsauto on Thursday January 30, 2020 @11:03PM (#22198927)
A system with more inherent friction -- whether cognitive or social expectations -- will be out-optimized by one with less.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by jchw on Thursday January 30, 2020 @06:36PM (#22197207)
Actually I think individuals suffer so that less SPAM gets through. The SPAM is flowing regardless.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigtunacan on Thursday January 30, 2020 @03:43PM (#22195326)
Please explain the difference to me. Mailgun is sending tons and tons of spam email; what is so special about them that they aren't ending up in people's spam/junk folders?
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by aseipp on Thursday January 30, 2020 @05:14PM (#22196383)
Mail providers like Mailgun live or die based on their reputation. If they as a sender get marked as spam by "The Algorithms", then their product is useless. They almost certainly respond to abuse reports quickly and have enough reputation, and enough legitimate clients, that they have "leeway". But that leeway isn't unlimited.

There was a big fit thrown on this website a year or two ago when one transactional mail provider (MailChimp?) said they were banning crypto companies with ICOs from their service. They literally put "ICOs are not welcome" in their Terms of Service. Why? Because first off, ICOs are scams. And that means they buy email lists, and then spam the shit out of people with "buy into our ICO!" emails, which get immediately shitcanned into the spam folder by 99.99% of users -- they're no different than Nigerian Princes, as far as most people care. That behavior tanks the reputation of the sender, and you cannot reason with The Algorithm once it "recognizes" you as spam. That kind of problem is not a minor inconvenience to companies like Mailgun, it's an existential threat.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by lolc on Thursday January 30, 2020 @07:59PM (#22197873)
They really care with their reputation on the line. Here for example is their response to the only complaint I've had to send Mailgun: "Thanks for sending this over. I was able to tie this sender to a compromised account, which has been disabled in our system to prevent further messages."

Pity we'll have to move off them. It was good while it lasted.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by slig on Thursday January 30, 2020 @04:29PM (#22195885)
They probably send much, much more valid emails, respond quickly to abuse reports, and since they're large/known enough, they won't get blacklisted because that would be disruptive to large, legit business.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mirimir on Thursday January 30, 2020 @10:39PM (#22198808)
That cuts both ways.

You might miss something valuable. Perhaps from a long-lost relative. Or a prospective client or employer.

And once you're actually looking through the spam folder, what good is it really?

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by gwittel on Thursday January 30, 2020 @04:45PM (#22196075)
Sending at scale is the problem. A provider like Mailgun (aka an ESP), is a huge target for abuse. Free and low priced tiers are tricky since they need to draw in customers, but then get heavily abused.

tl;dr: Abuse ruins it for everyone.

I work on the filtering side of the world so I see a lot of the challenges that even purportedly good actors face. Each provider does better or worse at controlling abuse. These things all have a cost:

* IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers).

* Dmarc/dkim/spf -- Passing validation can be hard since it requires your client to work with you.

* Abuse reports -- Likely huge volume

* Bounces -- They can count against the customer. How much is OK (e.g. AWS SES will cut you off if your bounce rate gets too high)?

* Throwaway account abuse -- Huge issue, this threshold is a simple hammer (so now bad guys need ~10x more throwaway accounts).

* Account takeover -- That good guy is now sending out crap. Now what?

* Post-abuse cleanup -- Good luck going around and working with major providers to get yourself unblocked. Its a huge time sink.

By using Mailgun, etc all of the above becomes their problem. On top of it they'll offer analytics, help crafting content that works in varying mail clients, etc.

A small business type sender doesn't need much of this (other than maybe the technical help). But at some point, the scales tip.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by webbie917 on Thursday January 30, 2020 @07:41PM (#22197729)
^ Couldn't agree more, especially "Abuse ruins it for everyone."
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by quicksilver03 on Friday January 31, 2020 @04:35AM (#22200312)
> IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers).

I'm curious, is that specific to IPv4 or you see the same phenomenon with IPv6 where providers allocate /64 or /48 to customers? Or there simply isn't enough sending volume from IPv6 at the moment to warrant specific filtering?

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by gwittel on Friday January 31, 2020 @09:15AM (#22201791)
It’s mostly IPv4. The volume over v6 isn’t there.

Blocking via IP on V6 is tricky since it’s not clear what a safe/useful block range would be.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by cx42net on Monday February 03, 2020 @08:10AM (#22223797)
You are right, most VPS provider for instance, don't give just one IPv6, but a full range. One might be able to implement a spamming server with rotating IPv6.

Spamhaus has released a "blocking strategy" for IPv6 to exactly address this issue: https://www.spamhaus.org/news/article/668/spamhaus-releases-...

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by cosmotic on Thursday January 30, 2020 @03:41PM (#22195315)
Most ISPs block SMTP traffic, which creates a pretty significant barrier to sending email. Even VPS services send through a secret gateways with all sorts of filters and controls on it.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by space_ghost on Thursday January 30, 2020 @03:35PM (#22195256)
You're both right, but you're solving different problems. Just because your message got past your spam filter doesn't mean it would get past all spam filters.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by kick on Thursday January 30, 2020 @03:41PM (#22195314)
It is absolutely not. Even if you are not considering the whole spam thing.

They specified that it wasn't even if you could ignore the spam issue. I said that it was if you ignored the spam issue. The first post in this thread was "We've made sending electronic mail way too complicated under the guise of fighting spam."

I'm aware that it won't get past every spam filter, but it gets past both Gmail's and my own provider (a smaller, non-American one), which is good enough for my own use. My claim was that sending electronic mail is a solved problem, because it is.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by edmundsauto on Thursday January 30, 2020 @06:01PM (#22196907)
Similarly, talking is a solved problem. Talking, so that people will listen, is not.

Communication is much more than just broadcasting, just as email is about more than sending emails.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigiain on Thursday January 30, 2020 @05:11PM (#22196356)
> My claim was that sending electronic mail is a solved problem, because it is.

Agreed. I mean, I can _probably_ remember enough to be able to send mail using just telnet without even needing to look anything up. (And that mail, without any mime parts or urls - it very very likely to not fall into any of the spam filtering on my inbound email accounts...)

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by greglindahl on Thursday January 30, 2020 @07:30PM (#22197650)
In the modern era, lots of mailservers run software that looks for weirdnesses in incoming email. I don't know, but I'd suspect that the typing delays of using telnet to send mail are likely to trigger their defenses.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigiain on Thursday January 30, 2020 @09:11PM (#22198350)
SO I just tested. After working out that on macOS I now need to use nc -c instead of telnet... I _can_ send mail manually, but it _does_ end up in gmails's spam folder saying "Why is this message in spam? It is similar to messages that were identified as spam in the past."

  Iain-2:~ iain$ nc -c gmail-smtp-in.l.google.com 25
  220 mx.google.com ESMTP k5si5682618pls.209 - gsmtp
  EHLO elided.org
  250-mx.google.com at your service, [202.171.181.100]
  250-SIZE 157286400
  250-8BITMIME
  250-STARTTLS
  250-ENHANCEDSTATUSCODES
  250-PIPELINING
  250-CHUNKING
  250 SMTPUTF8
  MAIL FROM: <[email protected]>
  250 2.1.0 OK k5si5682618pls.209 - gsmtp
  RCPT TO: <[email protected]>
  250 2.1.5 OK k5si5682618pls.209 - gsmtp
  DATA
  354  Go ahead k5si5682618pls.209 - gsmtp
  Subject: Test
  From: [email protected]
  To: [email protected]
  Date: Fri, 31 Jan 2020 13:07:00 +1100
  
  Foo baz nah
  
  .
  
  250 2.0.0 OK  1580436443 k5si5682618pls.209 - gsmtp
  QUIT
  221 2.0.0 closing connection k5si5682618pls.209 - gsmtp
  Iain-2:~ iain$
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by wolco on Thursday January 30, 2020 @10:12PM (#22198673)
I wonder why google treats this simple email as spam. It contains nothing that should trigger the spam filter aside from where it came from.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by greglindahl on Thursday January 30, 2020 @10:54PM (#22198880)
Most mailservers don't send emails one character at a time. You appear to have a mental model of how a spam filter works that doesn't include fingerprinting the behavior of the sending server? Even open source mail servers do that, for example postscreen is a part of postfix.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigiain on Thursday January 30, 2020 @11:24PM (#22199036)
I wonder if it'd fare better if I scripted nc with expect?

(Not sure I care enough to try...)

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by nrb on Friday January 31, 2020 @04:26AM (#22200271)
Where it came from is one of the most important details!

Is this mail relay actually elided.org or is it impersonating elided.org?

There does not appear to be an SPF record for that domain so Gmail cannot determine if this is a fraudulent message, especially if the machine this request came from does not match the IP of the A record for that domain.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by duskwuff on Thursday January 30, 2020 @11:08PM (#22198953)
Some of the things it doesn't contain, though -- like Message-ID and Content-Type headers -- are likely to contribute to its spam score.

If your domain has SPF headers which don't permit mail from the (residential?) IP you're connecting from, that'll be a major factor as well.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by arein2 on Friday January 31, 2020 @02:39AM (#22199857)
Some years ago I wrote my email in a forum, available to crawlers. I’ve been getting spam ever since.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by NightlyDev on Thursday January 30, 2020 @07:19PM (#22197569)
I've sent a lot of mail in the last 14 years, and it's not hard.

I spammed with invite mails to users friends back in 2008 and mail started to end up in spam for a while, but after switching to double opt-in for all mail and making unsubscribing easily available it went back to good. Add some periodic list cleaning of long time inactive subscribers and it works perfectly!

It does get harder if you have to account for bad users, but if you're the only user then you can make sure everything is good.

So, is it hard? That depends entirely on how wanted or unwanted the sent mail is.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ruffrey on Thursday January 30, 2020 @03:44PM (#22195333)
I've been running a SaaS in the email space since 2011. It is not surprising at all, to me, that Mailgun decided to do this.

Sending an email _is_ trivial - you're right - but getting it delivered to the inbox, instead of the spam folder, is far from trivial. At scale, it is incredibly difficult. Even worse, spammers and fraudsters are utterly relentless. They hammer services like mine and Mailgun's constantly with fraud and phishing and porn and other garbage, inbound and outbound.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by nemothekid on Thursday January 30, 2020 @04:38PM (#22195989)
>despite sending an email being an utterly trivial, long solved problem

AFAIK, SPAM is a still a hard problem, and getting emails delivered isn't trivial.

"Under the guise of Spam" is a very reductionist viewpoint. It's like saying "Under the guise of the measles, we now have an oligopoly of pharmaceuticals." The latter might be true, but its important not to understate the threat of the former.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by xwowsersx on Thursday January 30, 2020 @03:15PM (#22195042)
It is clearly not trivial though. I get that it feels like it should be and probably is if you're sending a few emails, but "at scale" (pardon the buzzword) it is not trivial hence why people use these services.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ttul on Thursday January 30, 2020 @06:46PM (#22197290)
If it's an oligopoly and presumably throwing off cash, what's stopping anyone from creating their own transactional email service?

Answer: It's very difficult. Email is adversarial. Getting to the inbox is incredibly difficult to do _at scale_.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by NightlyDev on Thursday January 30, 2020 @07:32PM (#22197661)
A private service where you are very sure that subscribers wants and opens the mail? Easy.

A public service where a lot will be spam and unsolicited emails? Must be hell to run.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by luckylion on Thursday January 30, 2020 @05:55PM (#22196846)
No, we've created something far better. ~50% of the Spam that makes it to my inbox is sent via Amazon SES. Amazon does not give a fuck, I've reported it to abuse, they ignore it.

I can't just block Amazon SES though, because there's also a ton of legit services that use them for mail transport. We've really helped spammers evade blocking and decrease their infrastructure costs, and because there's so much "normal" traffic on SES, spammers will blend right in. And as long as they pay for services rendered, Amazon couldn't be happier.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ljm on Thursday January 30, 2020 @06:18PM (#22197052)
These providers are just as responsible for spam by running opt-out mailing lists and, even worse... spamming you with surveys and confirmations to see why you unsubscribed. With tracking pixels and rewritten URLs that go through their system before you get to the real destination.

You don’t get a choice. That was made for you.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by omarhaneef on Thursday January 30, 2020 @04:28PM (#22195865)
Yes, sending email is indeed a solved issue, but that isn't what mailgun (re?) solves.

Asking someone's permissions first, tracking their desires and immediately updating your response is what they solve.

Seems trivial? Try doing it for (looks at note) 626 people.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by hnick on Thursday January 30, 2020 @06:11PM (#22196990)
I think their point was that it's been drummed into our heads that ISPs and mail services only accept email from 'trusted' IP blocks, so you can't do it yourself, better use 'the experts'.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by pyuser583 on Friday January 31, 2020 @05:04PM (#22206349)
You want to try an impressive open source software project? Create a framework that takes an email forwarded to a given address, turns it into an HTTP request with a JSON body, and sends that HTTP request to webhook.

It’s easy to make a proof of concept. Production grade is an entirely different story.

Remember to preserve conversations and forwarding. Make sure file attachments are handled and scanned for viruses. Make sure you watch out for recursive automatic forwarding. Make sure that any html is secure. And make sure it will scale well ... and I mean billions of emails a day well.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by cx42net on Monday February 03, 2020 @08:12AM (#22223818)
You are 100% right. It started the same with our service; we bought back ImprovMX (I don't put the link as I'm not looking for publicity - just for reference) when it was just a 5$ droplet at Digital Ocean, and now we have 10 big servers, more than 700$ monthly cost (and I honestly think it's not that much) and have to handle all the messy parts of delivering an email.

It's easy to send an email, but it's way harder to ensure that the emails you send are delivered properly.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mike-cardwell on Friday January 31, 2020 @05:54AM (#22200665)
We have our own domain in G-Suite for our mail. I just checked my GMail Spam box. In it are 6 emails from this week. All 6 of them were sent through Mailgun, from and to the same domain (the one in G Suite). One was a cron email from one of our servers and the other 5 are transactional emails about people signing up to our service.

All of them passed both SPF and DKIM.

How. Just how did these emails get spam filtered by Google? Everything about them screams that they're legit emails from ourselves to ourselves.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by Keverw on Monday February 03, 2020 @03:14AM (#22222361)
Wow, sounds like even if you pay for a company to do then still might not go to inbox 100%... I noticed marking things not as spam still sends things to spam, I figured be doing that would tell it it's not spam...

I was recently thinking about the email problem myself, and was reading about IP Warmups. Seems like every provider has different recommendations too. So sounds like not an exact science either. But maybe if it was an exact science it'd be hard to tell the good and bad guys apart. I guess one of the tricky problems with decentralized systems.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bdcravens on Thursday January 30, 2020 @07:07PM (#22197482)
> despite sending an email being an utterly trivial, long solved problem

Then why do companies need transactional email providers?