News for hackers. Stuff that matters.
Re:Companies may be losing millions due to emails buried in ... by tzs on Thursday March 14, 2019 @11:06AM (#19389856)
I worked at a place that had many customers who used a major free email provider that also ran an ad network (but was not Google or Microsoft) that kept deciding that most mail we sent to our customers was spam: receipts, installation instructions, responses from support, and notices that subscriptions were going to re-bill were all blocked.

We'd contact them, they would fix things, and it would work for a while...but invariably it would go back to classifying it all as spam, and we'd get customers calling to complain about not getting instructions or support responses.

Finally, our guy who managed our ad spending called up our rep in ad sales at the email company, and asked in rather forceful terms just why we should continue spending $large_number/month on ads there to acquire new customers, when we were then going to be blocked from emailing those customers?

The ad rep conference called in their head of IT, who conference called in their engineer in charge of spam filtering, and had him right there add us to a whitelist that would prevent anything from us being classified as spam no matter what the rest of their spam system decided. As far as I know we never had another email go into the spam filter.

If we hadn't been spending a large amount on ads with them, I doubt the problem would ever have been resolved.

If you sell a software product to end users, and have non-marketing messages that you need users to see such as responses to support requests or reminders that subscriptions are going to automatically re-bill, I'd recommend including some kind of messaging system specifically for this in the product itself. Still send them as email, but also send them via this in-product channel, or at least send notices that there is a message with links to a way to view it on the web. Email only is simply not reliable enough.

Re:Companies may be losing millions due to emails buried in ... by pmlnr on Thursday March 14, 2019 @11:10AM (#19389902)
> Email only is simply not reliable enough.

That is because implementations, especially gmail, ignore the specifications: (EDIT) there are specific error codes and messages should be sent back to the originator. The reasons to ignore it is, in theory, not to give spammers ideas how to avoid it.

Email is incredibly reliable, it can tolerate outages, configuration errors, delays, etc, by specification, but ever since "modern" spam filtering, gmail (and those who follow their practices) it became a nightmare, because everyone ignores how it _should_ work. It was done in a time when connections were slow and prone to errors; it had to be flexible.

Yes, I know spam was bad I was there. But I'd rather have spam, than actually important mails getting dropped because of false positive, paranoid spamfilters.

Re:Companies may be losing millions due to emails buried in ... by mikeash on Thursday March 14, 2019 @12:08PM (#19390550)
When there’s too much spam, your important mails are effectively dropped as they’re buried in the noise. Modern spam filtering is what saved email. Without it, you’d never see those important messages, because new spam would come in faster than you could delete it.
Re:Companies may be losing millions due to emails buried in ... by jstanley on Thursday March 14, 2019 @02:57PM (#19392327)
I don't have any content-based spam filtering (but I do use the spamhaus sbl-xbl list).

It doesn't take very long to manually delete spam from the inbox. And if your spam filter is putting legitimate emails in your spam folder, you need to manually filter them anyway.

Re:Companies may be losing millions due to emails buried in ... by mikeash on Thursday March 14, 2019 @03:24PM (#19392622)
At the peak of my spam experience, I was getting hundreds a day. That was ages ago when the internet was much slower and more expensive than it is today, and bad people online were much less sophisticated. I wouldn’t be surprised if a totally open email system today generated enough spam that even deleting one message per second you’d fall behind.

I haven’t checked my spam folder on a routine basis in many years.

Re:Companies may be losing millions due to emails buried in ... by jstanley on Thursday March 14, 2019 @03:37PM (#19392770)
What I'm saying is that "modern spam filtering" (by which I assume we mean GMail-style highly-aggressive spam filtering that puts lots of legitimate email in the spam folder) is not what saved email. Rudimentary spam filtering saved email. There's no need to ever have a legitimate email go to the spam folder.
Re:Companies may be losing millions due to emails buried in ... by T-hawk on Thursday March 14, 2019 @06:16PM (#19394397)
The definition of spam changed. Originally, it was scattershot bulk email, where the sender would just send to any address they had ever acquired for any reason.

Now spam means "anything the user doesn't want to receive". Even if the user previously deliberately created a relationship with the sender. Gmail successfully trained users into clicking the spam button for anything such. People just click Gmail's magic make-it-go-away button for things as legitimate as a mailing list they deliberately signed up for and are simply too lazy to properly unsubscribe.

Legitimacy isn't a binary either-or, it's a continuum. You bought from a merchant ten years ago and they email you again, is that spam? How about the company that acquired that merchant which is now trying to sell you a completely different line of business? You donated to a politician and now their successor in the same party emails you, is that spam? That's what modern spam filtering deals with.

Re:Companies may be losing millions due to emails buried in ... by perl4ever on Friday March 15, 2019 @07:29PM (#19405307)
'Now spam means "anything the user doesn't want to receive".'

Spam means anything from someone who is indifferent to whether the recipient wants to receive it and doesn't get "sufficient" permission. I think this is consistent with the original type of spam, even if there is no bright line delineating it.

If you're implying that anything with a working unsubscribe link shouldn't be considered spam, then I think this definitional issue relates to the same controversies about what constitutes consent that have been discussed a lot recently.

I mean, I agree with your last paragraph, so far as that goes, but I would suggest rather than the definition of spam changing, ordinary businesses got spammier/sleazier.

Re:Companies may be losing millions due to emails buried in ... by user5994461 on Thursday March 14, 2019 @04:36PM (#19393410)
Rudimentary spam filtering saved email long ago when spams were focused on a few niches like pharmaceutical or Nigerian scam.

It wouldn't be efficient today with spam being more more diverse and often hardly distinguishable from legitimate emails. A lot of spam nowadays is like forced advertising, by opposition to plain scam and dodgy stuff.

Re:Companies may be losing millions due to emails buried in ... by efaref on Thursday March 14, 2019 @03:55PM (#19392948)
Spam filtering is an arms race. Spammers know how to download all the existing spam filtering software and tweak their emails to make sure it gets past it.
Re:Companies may be losing millions due to emails buried in ... by joveian on Thursday March 14, 2019 @05:19PM (#19393860)
My experience matches jstanley's: with basic ISP filtering and disabling as much filtering as they let me disable my ISP reports blocking 181 messages in the past month mostly via some list (they don't say which one). I see a few spam messages per day and generally get half a dozen or more copies the first time I see a new one, conveniently making them easy to delete (and very few have confusing subjects). I'm fairly sure I see less than half as many as the ISP blocks. This is with an address that has been public for over a decade. It doesn't seem like much of an arms race to me and it is a trivial effort to delete them.

The only things I can think of that I do differently than many people that might make a difference is that I have my mail client set up to not show remote images ever and I never dirctly click any link from email no matter the source. Possibly things might be much worse otherwise, but from my perspective it doesn't seem like spam is a huge issue these days with basic filtering.

Re:Companies may be losing millions due to emails buried in ... by mikeash on Thursday March 14, 2019 @07:50PM (#19395096)
You're still benefitting from everybody else's more sophisticated spam filtering. The fact that most users have good filtering means that the cost/benefit ratio of sending spam is much worse than it otherwise would be. This naturally decreases the activity significantly. Saying that you get by with just basic blocking is the e-mail equivalent of saying that you don't need a measles vaccine. It may be true but it doesn't tell us much about what the world would look like without those measures.
Re:Companies may be losing millions due to emails buried in ... by joveian on Thursday March 14, 2019 @10:20PM (#19395882)
Where is the evidence that this is true? It doesn't seem that likely to me that so few people are not using gmail-style spam filtering that spammers couldn't figure out how to target those of us who aren't (the filtering that I could have used but turned off is more random than helpful and I would guess this is the case on many non-Google mail providers). Even with Google running a huge portion of all email there are still a huge number of people with email not provided by Google.

An alternative narrative is that previously email servers were supposed to accept almost anything from everyone and spammers took advantage of this. From this perspective it is the relatively recent requirement of sender verification that made the key difference and allows Spamhaus to work (obviously they need to do something to determine who to block). This perspective seems to match what I am seeing.

Re:Companies may be losing millions due to emails buried in ... by krageon on Friday March 15, 2019 @03:32AM (#19397427)
> It doesn't take very long to manually delete spam from the inbox.

I have tuned the spam filters for a fairly major email provider for a while and while you are right for a lot of people, there's a significant fraction of them that receives hundreds or even thousands of spam emails per day. For them, it is not practical - and everyone needs to be able to use it.

Re:Companies may be losing millions due to emails buried in ... by anitil on Thursday March 14, 2019 @07:36PM (#19395005)
I have subscribed to more email lists these days than ever before. Why? Because trust in the system has increased. I know that I can easily subscribe/unsubscribe and that decision will be respected (or punished if ignored).

So while gmail has made life harder for people who want to email me, it have made life much easier for me as an email user - which makes me more willing to give out my email.

Re:Companies may be losing millions due to emails buried in ... by lostlogin on Thursday March 14, 2019 @11:23AM (#19390022)
You’ve just explained why it’s become unreliable. It might have been great once, but fixing it when it’s broken is a complete nightmare now.
Re:Companies may be losing millions due to emails buried in ... by EGreg on Thursday March 14, 2019 @11:39AM (#19390214)
Email

SMS

Notifications

They all go through some provider who can block you.

It’s a rare product that people check on their own without notifications!

Any ideas how to reach people without gatekeepers? I have just one way: a desktop app or mobile app that is allowed to periodically wake up and poll a server.

Re:Companies may be losing millions due to emails buried in ... by cr0sh on Thursday March 14, 2019 @12:50PM (#19391018)
> Any ideas how to reach people without gatekeepers? I have just one way: a desktop app or mobile app that is allowed to periodically wake up and poll a server.

If you're asking "anybody today" then yes, that's probably one of the only ways.

But maybe tomorrow might be better.

On the horizon (where it may stay - it's been slow in coming) is the idea of "distributed and serverless" internet. Also known as "peer-to-peer".

You can think of it as everyone (well, their device) being their own server - kinda like the internet was originally meant to be (before vast sums of money got involved).

Through "magic" - each person that is on the network can browse, send, receive, post, etc - to others without needing much of anything else - just the pipe (not really any way around that kind of "gatekeeper" unless you have a mesh network of some sort - and even there, long-haul can be tough or impossible). A well known example:

https://ipfs.io/

...there are others out there as well; just google around for "peer-to-peer web" and similar terms.

Re:Companies may be losing millions due to emails buried in ... by EGreg on Thursday March 14, 2019 @01:10PM (#19391227)
Funny that you mention that. One of my company’s initiatives is building social networking software to run on local networks and mesh networks. I also want to see a future where cellphone signals, energy generation and so on don’t go through “last mile” gatekeepers but those are just one of many commodity choices to connect to “the grid” / “the cloud” / “the mesh” whatever it is called.

I think that is super important given what’s going on:

https://qbix.com/blog/2019/03/08/how-qbix-platform-can-chang...

We have started to work on captive portals so your device can surf from wifi to wifi just like a cellphone does with towers, and check in wherever you visit without you doing anything (with your prior permission, of course)

https://qbix.com/blog/2017/12/18/power-to-the-people/

IPFS is great, as is BitTorrent. I have met a lot of the people running these projects, like Tom Berners-Lee and the guys from Solid (back then), David Irvine and the guys from MaidSAFE, and Petar Maymounkov from NYU who invented Kademlia DHT.

I think there are several things that are needed:

  NAT hole punching
  End to end encryption
  Kademlia or other DHT
  Small group consensus
  Merkle DAG with validators
All these can be incrementally added to an existing, web-based solution. The problem with BlockStack and MaidSAFE is that they require you to download a new browser that only works on the desktop. Instead of trying to build on the existing Web.

For me it’s not even so much about privacy as being permissionless. No gatekeepers.

Whether it’s your cellphone company or your cable internet ISP or the Google Maps API or Amazon Web Services, they require you to pay them for their closed-source infrastructure and data.

The infrastructure should be a mesh and the data should be open source. The challenge is simply in the software to coordinate the mesh participants, and for the data it’s software to enable the right rules to maximize the chances of data being correct.

There is tons, tons to do and frankly most VCs and investors don’t get it. The payoff is huge for humanity as a whole, and can unleash even more innovation than the Web did after it disrupted AOL etc. But they don’t capture even 10% of the whole value. It’s free to the world.

So we’re chronically underfunded but we have already put together so much. And now we are making money the old fashioned way: earning it from users and clients hahaha. The Basecamp guys would be proud.

Re:Companies may be losing millions due to emails buried in ... by stdcli on Thursday March 14, 2019 @03:34PM (#19392731)
Blockstack Engineer here. You do not need to download a browser to use Blockstack. We also have a working mobile version, and multiple apps built on the Blockstack API that have their own ios and android apps. You can check out Blockstack without downloading the browser here: https://browser.blockstack.org

If it is your first time, you will be asked to create an identity on Blockstack: https://docs.blockstack.org/core/naming/introduction.html

Each instance of the "browser" whether on desktop or phone, requires a unique password, associated with the id. The id is stored on the Blockchain, the data is stored in your own gaia hub which you can host wherever you like: https://docs.blockstack.org/storage/overview.html

In regards to your comment about "Kademlia or other DHT": Blockstack previously used Kademlia, but iterated on an improvement to Kademlia by creating Atlas which is what we use at Blockstack. More here: https://github.com/blockstack/atlas

and the specific reasons about why Atlas was built in lieu of Kademlia here: https://blog.blockstack.org/blockstack-core-v0-14-release/

Together, Atlas, BNS, and gaia create the foundation upon which the Blockstack API is built on.

You can build applications using the Blockstack API here: https://docs.blockstack.org/develop/zero_to_dapp_1.html

If you want to explore the ecosystem of existing decentralised applications on Blockstack and other decentralised solutions, you can see that here: https://app.co/

You can read more about how identity, Atlas, and user owned storage works here: https://blockstack.org/whitepaper.pdf and maybe poke around here a bit: https://core.blockstack.org/

Oh yes, also you can download the browser on your desktop if you like.

Re:Companies may be losing millions due to emails buried in ... by EGreg on Thursday March 14, 2019 @06:14PM (#19394381)
Hey, are you around in NYC? Would love to grab coffee with you and discuss archtecture together. I am curious how Kademlia and DHT turned out to be unreliable in practice and the lessons you learned for why what you use now is better

Can you drop me a line at username greg with the domain qbix.com ?

Re:Companies may be losing millions due to emails buried in ... by stdcli on Thursday March 14, 2019 @06:37PM (#19394589)
Absolutely. We will be in touch shortly. In the mean time, you can check out our forum: https://forum.blockstack.org/ or hit is up on our slack.
Re:Companies may be losing millions due to emails buried in ... by EGreg on Saturday March 23, 2019 @08:48PM (#19473298)
Hey, didn’t get any email from you
Re:Companies may be losing millions due to emails buried in ... by EGreg on Sunday March 17, 2019 @03:51AM (#19412797)
Cool — what to look out for, to make sure I don’t miss it?
Re:Companies may be losing millions due to emails buried in ... by nasredin on Thursday March 14, 2019 @04:43PM (#19393491)
Since you are name dropping projects:

Tox, a P2P Skype alternative.

Unfortunately it is not well known outside of tinfoil hat community.

Re:Companies may be losing millions due to emails buried in ... by ams6110 on Thursday March 14, 2019 @11:58AM (#19390424)
A printed invoice in an envelope and sent via Post mail usually still works.
Re:Companies may be losing millions due to emails buried in ... by daveFNbuck on Thursday March 14, 2019 @12:27PM (#19390773)
Then we have the problem of a service that doesn't sort out spam. I often get enough junk mail that it's hard to fit everything in my mailbox. Important mail ends up shoved between pages of catalogs I'm about to throw out or mixed in with an ever-growing stack of junk that I'll sort through later when I find the time.
Re:Companies may be losing millions due to emails buried in ... by imglorp on Thursday March 14, 2019 @04:44PM (#19393504)
It's a DOS attack on your mailbox. The SNR is approaching 0.
Re:Companies may be losing millions due to emails buried in ... by EGreg on Thursday March 14, 2019 @01:37PM (#19391501)
You’re kidding, right? The Postal Service is the ultimate gatekeeper. You have only a companies that would make the last mile delivery.
Re:Companies may be losing millions due to emails buried in ... by lixtra on Thursday March 14, 2019 @02:49PM (#19392238)
> wake up and poll a server

Here the AppStore is the gatekeeper. OP is probably right, your website is the most reliable place to communicate with your customers.