News for hackers. Stuff that matters.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by Eikon on Thursday January 30, 2020 @02:59PM (#22194867)
> despite sending an email being an utterly trivial, long solved problem.

It is absolutely not. Even if you are not considering the whole spam thing.

Sending an email with the multitude of clients, MIME which is very complex in itself, network issues, message queuing, retries, unsubscription, bounces, providers feedback loops, rate limiting and what not is very hard.

If you add the whole spam folder + blacklists situation into consideration, sending emails becomes a more than painful thing you don't want to deal with.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by kick on Thursday January 30, 2020 @03:29PM (#22195192)
Sending an email is absolutely a solved problem.

I can send a message to my address in 2020 using a 2003 Fedora Core 1 box without a domain using only what's part of the POSIX standard, and I'll receive it just fine.

I just did a few days ago, actually. It even got past my provider's spam filter.

I could probably have done it using earlier software than 2003-era mail/mailx, but that was the quickest way I could find to send an attachment given the software on hand.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ebiester on Thursday January 30, 2020 @03:34PM (#22195246)
You can still do that.

However, I, as a receiver of email, get thousands of spam messages a day. So, I use services that block them before they even reach my view, or get sent directly to the spam folder.

If you want to send me an email, that's solved. If you want to send it to ten thousand people on your mailing list, and not have it end up in the spam folder, you may have more work.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by staticautomatic on Thursday January 30, 2020 @03:44PM (#22195340)
This raises what's really the ultimate question, which is whether and how much individuals should be made to suffer so that advertisers can send spam. Certainly the recipients are not responsible for the advertisers' decisions to spam, or more charitably, to blur the lines. Yet it's individuals who pay, in this case literally, for the advertisers' privilege of spamming them. It's extremely fucked.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by edmundsauto on Thursday January 30, 2020 @06:00PM (#22196898)
I think this is what happens in open systems. The alternative is walled gardens, like FB messages. Even those suffer from spam, although it's much less of the wild west.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mc3 on Thursday January 30, 2020 @09:47PM (#22198530)
The alternative is simple:

For stranger to stranger communication, both business and non-business set up a web form. With a challenge if needed.

For friend to friend communication, use email, friends' address in on a white list.

For business to consumer communication, white list is again used.

For spammer to anyone communication, including business lists that refuse to take you off, not on white list so doesn't get in. Bounce response with reason "non on white list"

It's a cultural change though: an email address won't get through unless you are unblocked, but technically easy.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by sokoloff on Thursday January 30, 2020 @10:12PM (#22198669)
What's the easy/low-friction way for Joe Random Consumer to manage their whitelists?

An email comes in from a sender address which isn't on the whitelist, but is (in fact) a friend. What's the handling?

An email comes in from a sender address which isn't on the whitelist, but is not (in fact) a friend. What's the handling?

Do I never hear from the first friend? Or do I get bombarded with "hey, read this email and see if the sender is a friend" a bunch of times?

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by ezrast on Friday January 31, 2020 @03:26AM (#22200024)
I'm just shooting from the hip here, but I think it's:

Joe manages his whitelist through his mail provider's web UI. Many providers already use the address book as a whitelist; the only difference is they default to filtration instead of assuming spam.

Joe's friend's mail gets routed to the spam box. Joe's friend gets a bounce notification that says "To be added to Joe's whitelist, click here (and optionally solve a captcha/enter Joe's dog's name/submit a blood sample/deposit $0.25 worth of bitcoin into Joe's wallet)." Joe's friend clicks here, and Joe's mail provider adds Joe's friend to the whitelist and promotes the email to the inbox.

Joe's spammer gets the same message and disregards it. Their email stays in spam forever.

Joe's spam box gets bombarded with "hey, read this email and see if the sender is a friend" a bunch of times. He can trawl through them for actual friends if he wishes.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mc3 on Thursday January 30, 2020 @10:23PM (#22198732)
Fair game. You got me, I have no nice answer other than phone your friend. I'll stop digging. I'm not going to do what this guy did: https://news.ycombinator.com/item?id=16862739
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by edmundsauto on Thursday January 30, 2020 @11:03PM (#22198927)
A system with more inherent friction -- whether cognitive or social expectations -- will be out-optimized by one with less.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by jchw on Thursday January 30, 2020 @06:36PM (#22197207)
Actually I think individuals suffer so that less SPAM gets through. The SPAM is flowing regardless.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigtunacan on Thursday January 30, 2020 @03:43PM (#22195326)
Please explain the difference to me. Mailgun is sending tons and tons of spam email; what is so special about them that they aren't ending up in people's spam/junk folders?
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by aseipp on Thursday January 30, 2020 @05:14PM (#22196383)
Mail providers like Mailgun live or die based on their reputation. If they as a sender get marked as spam by "The Algorithms", then their product is useless. They almost certainly respond to abuse reports quickly and have enough reputation, and enough legitimate clients, that they have "leeway". But that leeway isn't unlimited.

There was a big fit thrown on this website a year or two ago when one transactional mail provider (MailChimp?) said they were banning crypto companies with ICOs from their service. They literally put "ICOs are not welcome" in their Terms of Service. Why? Because first off, ICOs are scams. And that means they buy email lists, and then spam the shit out of people with "buy into our ICO!" emails, which get immediately shitcanned into the spam folder by 99.99% of users -- they're no different than Nigerian Princes, as far as most people care. That behavior tanks the reputation of the sender, and you cannot reason with The Algorithm once it "recognizes" you as spam. That kind of problem is not a minor inconvenience to companies like Mailgun, it's an existential threat.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by lolc on Thursday January 30, 2020 @07:59PM (#22197873)
They really care with their reputation on the line. Here for example is their response to the only complaint I've had to send Mailgun: "Thanks for sending this over. I was able to tie this sender to a compromised account, which has been disabled in our system to prevent further messages."

Pity we'll have to move off them. It was good while it lasted.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by slig on Thursday January 30, 2020 @04:29PM (#22195885)
They probably send much, much more valid emails, respond quickly to abuse reports, and since they're large/known enough, they won't get blacklisted because that would be disruptive to large, legit business.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by mirimir on Thursday January 30, 2020 @10:39PM (#22198808)
That cuts both ways.

You might miss something valuable. Perhaps from a long-lost relative. Or a prospective client or employer.

And once you're actually looking through the spam folder, what good is it really?

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by gwittel on Thursday January 30, 2020 @04:45PM (#22196075)
Sending at scale is the problem. A provider like Mailgun (aka an ESP), is a huge target for abuse. Free and low priced tiers are tricky since they need to draw in customers, but then get heavily abused.

tl;dr: Abuse ruins it for everyone.

I work on the filtering side of the world so I see a lot of the challenges that even purportedly good actors face. Each provider does better or worse at controlling abuse. These things all have a cost:

* IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers).

* Dmarc/dkim/spf -- Passing validation can be hard since it requires your client to work with you.

* Abuse reports -- Likely huge volume

* Bounces -- They can count against the customer. How much is OK (e.g. AWS SES will cut you off if your bounce rate gets too high)?

* Throwaway account abuse -- Huge issue, this threshold is a simple hammer (so now bad guys need ~10x more throwaway accounts).

* Account takeover -- That good guy is now sending out crap. Now what?

* Post-abuse cleanup -- Good luck going around and working with major providers to get yourself unblocked. Its a huge time sink.

By using Mailgun, etc all of the above becomes their problem. On top of it they'll offer analytics, help crafting content that works in varying mail clients, etc.

A small business type sender doesn't need much of this (other than maybe the technical help). But at some point, the scales tip.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by webbie917 on Thursday January 30, 2020 @07:41PM (#22197729)
^ Couldn't agree more, especially "Abuse ruins it for everyone."
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by quicksilver03 on Friday January 31, 2020 @04:35AM (#22200312)
> IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers).

I'm curious, is that specific to IPv4 or you see the same phenomenon with IPv6 where providers allocate /64 or /48 to customers? Or there simply isn't enough sending volume from IPv6 at the moment to warrant specific filtering?

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by gwittel on Friday January 31, 2020 @09:15AM (#22201791)
It’s mostly IPv4. The volume over v6 isn’t there.

Blocking via IP on V6 is tricky since it’s not clear what a safe/useful block range would be.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by cx42net on Monday February 03, 2020 @08:10AM (#22223797)
You are right, most VPS provider for instance, don't give just one IPv6, but a full range. One might be able to implement a spamming server with rotating IPv6.

Spamhaus has released a "blocking strategy" for IPv6 to exactly address this issue: https://www.spamhaus.org/news/article/668/spamhaus-releases-...

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by cosmotic on Thursday January 30, 2020 @03:41PM (#22195315)
Most ISPs block SMTP traffic, which creates a pretty significant barrier to sending email. Even VPS services send through a secret gateways with all sorts of filters and controls on it.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by space_ghost on Thursday January 30, 2020 @03:35PM (#22195256)
You're both right, but you're solving different problems. Just because your message got past your spam filter doesn't mean it would get past all spam filters.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by kick on Thursday January 30, 2020 @03:41PM (#22195314)
It is absolutely not. Even if you are not considering the whole spam thing.

They specified that it wasn't even if you could ignore the spam issue. I said that it was if you ignored the spam issue. The first post in this thread was "We've made sending electronic mail way too complicated under the guise of fighting spam."

I'm aware that it won't get past every spam filter, but it gets past both Gmail's and my own provider (a smaller, non-American one), which is good enough for my own use. My claim was that sending electronic mail is a solved problem, because it is.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by edmundsauto on Thursday January 30, 2020 @06:01PM (#22196907)
Similarly, talking is a solved problem. Talking, so that people will listen, is not.

Communication is much more than just broadcasting, just as email is about more than sending emails.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigiain on Thursday January 30, 2020 @05:11PM (#22196356)
> My claim was that sending electronic mail is a solved problem, because it is.

Agreed. I mean, I can _probably_ remember enough to be able to send mail using just telnet without even needing to look anything up. (And that mail, without any mime parts or urls - it very very likely to not fall into any of the spam filtering on my inbound email accounts...)

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by greglindahl on Thursday January 30, 2020 @07:30PM (#22197650)
In the modern era, lots of mailservers run software that looks for weirdnesses in incoming email. I don't know, but I'd suspect that the typing delays of using telnet to send mail are likely to trigger their defenses.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigiain on Thursday January 30, 2020 @09:11PM (#22198350)
SO I just tested. After working out that on macOS I now need to use nc -c instead of telnet... I _can_ send mail manually, but it _does_ end up in gmails's spam folder saying "Why is this message in spam? It is similar to messages that were identified as spam in the past."

  Iain-2:~ iain$ nc -c gmail-smtp-in.l.google.com 25
  220 mx.google.com ESMTP k5si5682618pls.209 - gsmtp
  EHLO elided.org
  250-mx.google.com at your service, [202.171.181.100]
  250-SIZE 157286400
  250-8BITMIME
  250-STARTTLS
  250-ENHANCEDSTATUSCODES
  250-PIPELINING
  250-CHUNKING
  250 SMTPUTF8
  MAIL FROM: <[email protected]>
  250 2.1.0 OK k5si5682618pls.209 - gsmtp
  RCPT TO: <[email protected]>
  250 2.1.5 OK k5si5682618pls.209 - gsmtp
  DATA
  354  Go ahead k5si5682618pls.209 - gsmtp
  Subject: Test
  From: [email protected]
  To: [email protected]
  Date: Fri, 31 Jan 2020 13:07:00 +1100
  
  Foo baz nah
  
  .
  
  250 2.0.0 OK  1580436443 k5si5682618pls.209 - gsmtp
  QUIT
  221 2.0.0 closing connection k5si5682618pls.209 - gsmtp
  Iain-2:~ iain$
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by wolco on Thursday January 30, 2020 @10:12PM (#22198673)
I wonder why google treats this simple email as spam. It contains nothing that should trigger the spam filter aside from where it came from.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by greglindahl on Thursday January 30, 2020 @10:54PM (#22198880)
Most mailservers don't send emails one character at a time. You appear to have a mental model of how a spam filter works that doesn't include fingerprinting the behavior of the sending server? Even open source mail servers do that, for example postscreen is a part of postfix.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by bigiain on Thursday January 30, 2020 @11:24PM (#22199036)
I wonder if it'd fare better if I scripted nc with expect?

(Not sure I care enough to try...)

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by nrb on Friday January 31, 2020 @04:26AM (#22200271)
Where it came from is one of the most important details!

Is this mail relay actually elided.org or is it impersonating elided.org?

There does not appear to be an SPF record for that domain so Gmail cannot determine if this is a fraudulent message, especially if the machine this request came from does not match the IP of the A record for that domain.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by duskwuff on Thursday January 30, 2020 @11:08PM (#22198953)
Some of the things it doesn't contain, though -- like Message-ID and Content-Type headers -- are likely to contribute to its spam score.

If your domain has SPF headers which don't permit mail from the (residential?) IP you're connecting from, that'll be a major factor as well.

Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by arein2 on Friday January 31, 2020 @02:39AM (#22199857)
Some years ago I wrote my email in a forum, available to crawlers. I’ve been getting spam ever since.
Re:Tell HN: Mailgun lowers free-tier API from 10k to 625 ema... by NightlyDev on Thursday January 30, 2020 @07:19PM (#22197569)
I've sent a lot of mail in the last 14 years, and it's not hard.

I spammed with invite mails to users friends back in 2008 and mail started to end up in spam for a while, but after switching to double opt-in for all mail and making unsubscribing easily available it went back to good. Add some periodic list cleaning of long time inactive subscribers and it works perfectly!

It does get harder if you have to account for bad users, but if you're the only user then you can make sure everything is good.

So, is it hard? That depends entirely on how wanted or unwanted the sent mail is.