I think this is what federation looks like when it works. Yeah, everyone uses Gmail, but it's also ok to use another service. It still works and delivers a pretty similar experience. Your messages make it out of the Gmail garden.
> Web sites were federated, but everyone being web master for their homepage was too difficult, so we get GeoCities, and Yahoo Clubs, and later, Squarespace and Wix, etc. Then blogs were decentralized, but everyone self hosting and authoring blogs was too much, so then we get Wordpress.com and Tumblr, and Twitter, and FB, etc. (even USENET eventually developed super-large hubs like uunet)
Geocities was one of many roughly equivalent services. It was sort of a social network, but it was mostly just free hosting. Anyone could still link to anywhere and being on geocities wasn't a requirement for being on the web.
Squarespace is a pretty good product and not what I would consider harmul centralization, because it's still interoperable with other websites and they don't prevent people from hosting elsewhere. The web is still federated.
Something different has happened with social media. Different social media services aren't interoperable. I can't message you on Twitter from Facebook, or tweet at you on LinkedIn. You must stay in their walled garden to interact with their users. They're at such a critical mass that almost everyone is in their garden.
It seems like federation is a happy middle ground between decentralization and centralized walled gardens.
Maybe something like Mastodon will some day become so popular that Twitter is forced to become part of the federation and compete on features and marketing instead of vendor lock-in.
With a federated version of any type of social media clone, eventually new entrants will prefer to join the federation instead of trying to make a new walled garden. It wasn't really an option for Hotmail to only let you interact with other Hotmail users, because the federation was already established.
There was tremendous demand for social media services, but no federation to provide those services when the demand hit and explosive growth happened, so now we have a centralization problem.
I was using s small British provider, and almost by accident discovered at some point in time that emails I sent to gmail addresses in the prior 2 months had been marked as spam, unless they were a reply something sent to me (so it wasn’t true that everything got lost - only stuff I originated).
I gave up and moved to FastMail because I don’t have the time to deal with this (and solving it once is no guarantee it won’t happen again)
So, email is federated but effectively not decentralized - unless you use one of the central services, you are at risk from being obliviously blackholed at any second (that happens to large providers too, but there’s safety in numbers - it will be discovered sooner by someone)
Looks like Google will have to keep the 5bn "fines for stupid things we do to EU citizens and companies" line on their budget for years to come.
I don't think this will fly when the relevant authorities finds out (a nice mix of protecting local business and punishing an annoying company that have ignored the rules too many times).
And frankly I think that is great!
Hoping Google will sooner or later return to their roots as Internet superhero.
Meanwhile get ready for more fines, I'm especially looking forward to the almost-destroying-browser-competition-on-desktop fine that I've been waiting for a while. : )
I've seen public entities switch to Office 365 in order to "get rid of spam" and thereby losing a lot of incoming legitimate mails in the same process. A fact that is worsened by users using the "Mark as spam" button to delete legitimate emails from their inbox (a practice that is horribly common).
Going from that to billion-EUR fines to Google and Microsoft is a very, very long way.
Companies need to validate email addresses.
I get AT&T bills, OnStar notifications, etc. I have never used any of their services. Someone else probably signed up with my email address.
Additionally, other companies are violating the CAN SPAM Act by hiding the unsubscribe ability behind a log in. Nextdoor does this.
You are, of course, absolutely right re. spam and rightfully so. On the other hand, I regularly get spam reports re. 100% legitimate mails, often in the middle of longer email conversations and so on, and where one side of the conversation marks entire email threads as spam.
We always follow up on spam complaints and the one thing we do get often in cases like that is that the other party only wanted to delete the emails and didn't realise there were other consequences.
Yes, there's a lot of people that knows how to use the "Mark as spam button". I still, however, maintain the position that a lot of other people don't know how to use such features.
I would unsubscribe and delete, but have not really thought about marking as spam or not yet. Just interested in some input there.
If I would get it from a company where I have not signed up the answer is of course obvious.
When I get the old "you'll be removed from our list within 10 days" message, I generally reply to it with a quick message: "Please remove me immediately or your next message will be marked as spam". Many times the sender has complied with a personal response provided someone saw my message, even for some large companies with "no-reply" emails.
For smaller companies - especially the mom and pop type operations - I'll usually give them a friendlier notice. "Hi, I unsubscribed and I'm still getting emails. Please remove me as I'd really rather not mark your messages as spam". I usually get a friendly personal response with apologies and a removal.
In all cases, after the company has been warned, I don't think twice about sending them to spam.
Then I discovered my provider didn't set up SPF and DKIM correctly so emails were failing authentication.
There are many factors affecting email delivery beyond SPF and DKIM, and it can be difficult for a small provider to get everything right.
So if you're having delivery issues I'd start with that, not by assuming without evidence that the big guys are intentionally classifying spam just because it's from a small provider.
Been running my own mail for a long time, and with one big exception[1] and occasionally spotting misconfigured systems, any time I've had delivery trouble it has been my fault.
If someone is bouncing you, make sure your ducks are in a row. One easy thing to do that won't catch everything but will help is to send mail to a Spamassassin-protected account, and look at how it classifies things.
[1] The Deathstar has been bouncing me for a long time, and their postmaster appears to be /dev/null. Given that I've had exactly one person using them I wanted to mail, they can bite me.
Except sometimes, like when Comcast looks at your DMARC policy that specifically says "p=none" and decides that means "reject all mail from this domain". Gotta love those guys.
Edit: s/spf/DMARC
It allows the owner of a domain to specify which mail servers are authorized to send mail from the domain.
Being small may not have been their problem. I run a single-user server and haven't had any trouble with delivery to Gmail.
It was only google, only non-response emails, so it is a google specific thing and not general IP reputation.
Switching to fastmail, otherwise exactly the same (domain, equiv dkim and spf setup) solved everything.
Sure looks like I was on the scoring edge, with fastmail being better (or some special trust arrangement) than my previous provider. Which is fine if I had gotten feedback and had a way to resolve this - but practically I didn’t.
Edit: I had a catchall on that domain forward to an account on gmail. It was suggested to me that may have been the problem. But i still do from fastmail, and everything is fine, so it’s more likely that overall reputation or special fastmail arrangements were the issue. Either way, i’m far from being alone with this issue - as evidenced even i this subthread. It is federated but not practically open to everyone, which is an implicit attribute most people associate with decentralization.
Same here. Perhaps I should switch to a larger provider too.
This is anti-competitive behaviour.
Especially clear since, as GP points out mail gets through if it is a direct reponse, meaning they knew very well it was a legitimate address.
This is abuse of bulk mail filter to squeeze out local competitors.
Even if they could kangaroo court it for standard business practices they wouldn't since they don't want to be known as "the dumb fuckers who banned domain based spam filtering" and face pressure from annoyed constituents now receiving piles of spam.
It seems clear from the description above that poster is someone recipient wants mail from.
As for the ignorant users using spam button I've seen that once since I started working.
Happy to do so, but which are the relevant authorities?
I tried searching for it though and this seems relevant:
https://www.gov.uk/government/organisations/competition-and-...
Microsoft, Apple. I’d move my mail to one of these to get away from Google but I can’t find a way to do so. I already pay for Office 365 and iCloud but I don’t get to bring my own domain for email. Unless I’m missing something.
I don’t pay Google for my email on my own domain because I got in early enough on Google Apps for your Domain or whatever it was called.
I’m happy to pay a reasonable price - I just don’t know whether it’s possible.
Fastmail is a relatively cheap option. Yandex mail will do it for free.
I've also tried Mastadon but their interface leaves something to be desired. The experience is not there yet.
I do agree on interoperability -- and while Zuckerberg is doing this with Whatsapp and Messenger, we should really take this across most messaging app.
I fear for increasing centralization ONLY because they have the resources and money to do this. This will not change unless we develop a consortium that has the numbers to make it work.
That's not true "federation" the way the OP meant. What's happened with email is that it's turned into an oligopoly: you can have email from Gmail, Apple, Yahoo (if you're elderly), or Microsoft, and that's it, unless you have a corporate address for work. If you want to run your own personal email server, forget it: you'll be considered a spammer and your emails will go into the bit-bucket.
Speaking of Facebook and whatsapp, a while ago I read that you couldn't post links to some other social networks. Is it true and if so which sites are affected?
Imagine your ISP bundling a full node together with your modem etc. That would be a start.
> I think one day we'll discover some ways to decentralize things in ways that resist re-centralization
Centralization unfortunately is still at the core, but we should strive to add more nodes so we can achieve more distribution. The more distributed the system, the less corruptible it would be.
As much as I hate censorship and blocking of content, I believe there is a place for it, for example, you don't want kids seeing pornography and the likes but I do hate it when they do it for political reasons, blocking campaigns or even conspiracy theories, I believe the mind should be able to explore and be critical of everything, this means being able to see/watch everything and analyze.
People say, that most people are connected by i don't know how many direct relationships (6?). The same may have been true for web pages.
When the search engines were much worse, people were using webrings, pingbacks, whatever - I remember going through lists of links at the end of paper magazines. Web was optimized for this kind of searching. It was called surfing for a reason.
Search engines like google hugely distorted the incentives on the web. In the name of seo and adsense people did crazy things to the web (as an ecosystem). There has been an entire cottage industry of individuals creating websites specialized just for getting pagerank for other websites, or for capturing audiences via other tricks (mirroring other websites or public/national databases), or for making the website a soup of boldened keywords.
Anyway, there are specialized searches all around, if you look. Product searches, torrent searches, Shodan, whois, transportation, pretty much anything that's annoying to search for in Google has a better alternative somewhere.
Can't say I see meaningful difference between DDG and Google on this anymore.
DDG keeps getting better and Google has been on a downward spiral for some time.
> It can find result using synonyms instead of the original words.
DDG does this as well. And unlike Google they aren't driving me crazy by ignoring my double quotes and verbatim searches.
I'm pretty sure the vast majority of email servers are corporate servers, not in FANG datacenters.
3-5 years ago I probably would have agreed with you. Today I'm not so sure. And even among the companies running their 'own' email servers, the physical hardware is very likely in a Microsoft/Amazon/Google datacenter.
Many are moving to Office365, and to a lesser degree, Gmail.
Microsoft is making it financially difficult to justify running your own Exchange on-prem.
This is why we’re hoping to steer Matrix from being decentralised-at-the-servers to being decentralised-at-the-clients.
We've seen this with OpenBazaar. It makes p2p pretty simple out of the box but if it's not in browser or a mobile app many won't even try it once.
I'm sure @kentonv has some ideas on what the main roadblocks to adoption are.
Bittorrent doesn't have a search engine, so of course they are not decentralized.
> yet a few trackers compared to clients
Well, not if you count the DHT (the decentralized tracker).
A worldwide Bionet, built from an amalgamation of semi-self-sustaining, solar-powered organics, electronics and EM/radio.
[0] http://www.bbc.com/earth/story/20141114-the-biggest-organism...
Well, federation is the sweet spot between distribution and centralization. I've run my own email service for quite some years now, it is just fine.
A lot of people have bad experiences running their own mail senders though, because they do not go through the checklists and deliverability tests to make sure things come through. Google's SMTP service will only accept mail over TLS, most good email services will reject mail from servers which do not have a matching rDNS on their IP, most will reject senders who have not set up SPF and DKIM, some will reject those which do not have a DMARC policy (including a reporting address).
If you use any good email deliverability testing tool, and iron these issues out in your configuration, you can fairly reliably deliver emails.
Currently media is downloading more than uploading even for video games. Client trust issues (and not allowing arbitrary cheating) encourage centralization to at least domain servers for the sake of validation and finding other players.
I think something ARGy like say Pokemon Go but strategic (you have fixed resource pools and open knowledge of everything thus cheaters can instantly be flagged as not playing the same rules). But that would also be niche.
One thing that really could use decentralization is IoT (along with actually making it worth useful and security).
Of course the whole domain is sadly a hot mess of greed often about gathering as much data from their customer, screwing them over with needless connectivity requirements and considering actual functionality an afterthought and security an afterthought to that and then wondering why it isn't catching on. That is a whole other rant.
I believe there are valid technical reasons for this, at least for coax connections; fiber not-so-much. At the same time, I get what you are saying.
...and this is how they want it. In short, they keep the "last mile" from allowing true "peering".
The best you can do - and this is to mainly get around the TOS statements that (technically) prevent you from running servers (as well as physically blocking you from running anything on port 80 and 25 mainly, plus probably others) - is to purchase a business plan instead of a consumer plan.
Though I am sure they will try to (or do) block this if the installation address is for a home and not an office (then again, what about small home offices and such?).
I don't know a good way around any of these issues, as it seems all high-speed providers do similar things (most are aimed at prevention of spam and other similar reasons). Even the WISPs I've seen in my area have weird terms (one seems really great for the rural area they serve - until you come to the clause about not being allow to run BT because of "multiple streams"; I tend to wonder if they only allow a single user to use a browser - on a single page - or what).
The chinese, russians, south koreans, japanese and even the EU are now looking to protect it's turf.
Wouldn't shock me if the internet truly becomes a network of national intranets. I suspect if the EU or any major player truly turns their nation into an intranet, it will accelerate the process with every nation controlling their own network and protecting their own interests, companies and information.
Though highyl unlikely, if china ever becomes a dominant player on the internet, I could even see us closing ourselves off to protect our internet.
I guess this is the price of success. It attracts political attention.
But not even professional sysadmins want to admin and maintain servers (at least for personal use).
Why several companies are trying to build self-driving cars but so few are building servers that you spin up and they're secure and work? We need "self-driving computers".
> Any decentralized system that is more efficient if centralized, eventually gets centralized by efficiencies of scale.
I’d add “or by network effects” since these aren’t “efficienciences of scale” (related to increased marginal value not decreased marginal cost) and are clearly at play with some (many!) centralized systems.
Based on the US military doctrine, we've got 5 dimensions: air, sea, land, space, and cyber. There are 4 dimensions designed by nature and 1 dimension designed by humans. And we're able to engage differently (e.g., at worst, becoming 'domains of warfare') in each of these dimensions.
Wouldn't it be neat to create a completely new 6th dimension, which is independent of the 5 we have, with distinctive features for its system? I'm curious if the brain-computer interface, like what Elon Musk is doing with Neuralink, could be that new dimension?
computer started as one processor and distributed terminals, then cpu spread to terminals, then back to server/clients, then p2p. Lastly consoles are threatened by centralized gaming datacenters ..
Email is federated, but everyone running and managing their own email servers is too costly, so for consumers, it migrated to large sites. Web sites were federated, but everyone being web master for their homepage was too difficult, so we get GeoCities, and Yahoo Clubs, and later, Squarespace and Wix, etc. Then blogs were decentralized, but everyone self hosting and authoring blogs was too much, so then we get Wordpress.com and Tumblr, and Twitter, and FB, etc. (even USENET eventually developed super-large hubs like uunet)
For a federated, decentralized system to work and resist centralization, it has to be the case that running a node is dead simple, cheap, and out-of-sight/out-of-mind. It also can't be the case that hosting on a more powerful cluster, colocated with other nodes, gives you large benefits or cost advantages, otherwise, it'll just get centralized again.
Even cryptocurrencies fail this. They have terrible efficiency, but at least they were supposed to be relatively flat, instead of centralized and hierarchical, but instead, a majority of the hashing power is owned by a few large entities, so in effect, back to large financial players controlling much of the power.
I think one day we'll discover some ways to decentralize things in ways that resist re-centralization, but in the mean time, beliefs that you'll achieve cyber/crypto-anarchy by clever protocol design and the federales won't be able to rubber-hose-cryptanalyze you, is a dangerous belief that diverts us away from demanding the government and society agree to the goals of freedom. If everyone wants unfreedom, underground internet usage is a slim consolation.