That was then. Those issues have been more or less fixed these days. XMPP is actually starting to look interesting again. For an example of a modern XMPP client (and discussion about the issues mentioned above) see here:
"Free" for CDN$2.89! Not that I care about the $3, but which is it, guys?
From the website:
How do I install Conversations?
Conversations is entirely open source and licensed under GPLv3. So if you are
a software developer, you can check out the sources from GitHub and use gradle
to build your APK file.
The more convenient way — which not only gives you automatic updates but also
supports the further development of Conversations — is to buy the App in the
Google Play Store.
Buying the App from the Play Store will also give you access to our beta test.https://f-droid.org/repository/browse/?fdfilter=conversation...
a) it was overly chatty (i.e. constantly sending/receiving information, draining your battery)
and arguably worse
b) it wasn't designed for changing addresses (wifi->data and back) or constant loss of connectivity.
So if you used XMPP some years ago you'd kill your mobile, fast, and would constantly lose messages.
Other features of modern IM solutions (say, an archive of your messages?) weren't present or usable either.
Today? Most of these issues are resolved. Prosody is a great server to get started (again) and comes with solutions for most (all? MAM aka 'Message Archive' is still somewhat scary imo) problems I mentioned above. Conversation is a very nice and modern XMPP client.
TL/DR: XMPP was 'bad' for mobile deployment for a long time and probably lost too much traction because of that. XMPP isn't bad in that respect anymore.
Federation has the problem of being harder to prevent abuse. And for systems that use the phone number as the user's identifier, like WhatsApp or Signal, it's even harder: which server is associated to an user? What to do if the user registers to more than one server? How to prevent a server from saying it has registered a user, when the user didn't do it?
And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations.
But is already solved. Explanations : "The State of Mobile XMPP in 2016" https://gultsch.de/xmpp_2016.html
> Federation has the problem of being harder to prevent abuse.
Precisely XMPP have already solutions for this problem
> And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations.
Answers here : "Objection to ‘The ecosystem is moving’" https://gultsch.de/objection.html
You might also interest by GnuNameSystem wich is a federate, decentralize and censorship resistance DNS https://gnunet.org/gns
However in reality opening a TCP connection to a server will silently drop and all the discussed behavior depends on the cellular provider sending an RST packet back when that happens.... which often doesn't.
So because google negotiates with carriers to not time out GCM connections those don't time out. Unfortunately other connections do, thus the power efficiency of other push based approaches on android.
Do you have any links for the "solutions for the abuse problem" part? They should make for interesting reading.
A solution is using address like greeting@name (meaning localpart@dnspart) https://linuxfr.org/users/apichat/journaux/salut-toto-saluta...
An other solution is the design of XMPP : you can control who can speak with you (the contact list).
Federation has the problem of being harder to prevent abuse.
But this is also an issue with centralized systems (see Facebook, Twitter right now).
And for systems that use the phone number as the user's identifier, like WhatsApp or Signal, it's even harder: which server is associated to an user? What to do if the user registers to more than one server? How to prevent a server from saying it has registered a user, when the user didn't do it?
This is "hard", but solved. Bitcoin and similar systems do this (prove identity) very well already.
And when you control the whole infrastructure, it's easier to provide a good user experience. With federation, you are depending on the quality of other's implementations.
Google doesn't control email, gmail is still a pretty good experience. Controlling the infrastructure should lead to better quality and experience, but practically often doesn't. Does anyone love iTunes?
Why do you think this is true? I can't see how federation is an answer to censorship. They would simply censor access to all the XMPP providers, just as they're censoring access to all the non-federated messengers.
Is your idea that people would spin up new providers so quickly that the censors wouldn't be able to keep up? Every time people switched, they'd have to rediscover their entire social network all over again, and there's an asymmetry between how difficult it is to get everyone over to different hosts while rediscovering where everyone is vs how easy it is for the censors to add a single line to their block list.
It'd be way easier just to have a centralized host and have people switch VPN providers as they get blocked, since then they don't have to rediscover each-other, but that isn't a great user experience either.
Just like with metadata hiding, really effective censorship circumvention is going to require new protocols and new techniques, so we're going to be more likely to see those emerge in centralized rather than federated systems (that are by their nature difficult to update). I think we'll be able to respond in Signal very quickly.
First we already know this situation with email, and we see that as a federate protocol email is not censorable. At least not as easy to censor as Signal is.
Secondly it's not possible because their is not a finish list of providers. By the way 1) everybody should-could be its own provider 2) everybody must use its own domain name DNS to make addresses like greeting@name.
Solution for 1) https://yunohost.org & https://wiki.debian.org/FreedomBox & http://modoboa.org/en
2) for this point see https://account.conversations.im/domain AND if the DNS is to centralize lets see for GnuNameSystem https://gnunet.org/gns for the address like greaing@name see https://linuxfr.org/users/apichat/journaux/salut-toto-saluta...
Just to be clear, this is never going to happen. We do not live in a world where computers are for "computer people" anymore. I've been running my own mail server since 1995, and I would not wish it on anyone.
Running my own mail server has not helped me with:
1) Metadata protection. Every email that I send or receive either has Google or Yahoo on the other end of it. Running my own mail server does not help me "own" my data at all.
2) Data protection. Federated protocols can almost never change (look at the history of IRC, XMPP, SMTP, HTTP). Email is stuck in time, which is why emails are still not encrypted. WhatsApp, on the other hand, had the freedom to deploy e2e encryption to over a billion people very quickly.
3) Censorship circumvention. Email is very easy to identify and filter using DPI. In a world where people are only doing host-based filtering, blocking 100 or 1000 hosts is no more difficult than blocking one. Whether or not there is "a finished list," if people can discover these services, so can the censors. It is much easier for the censors to add a single line to a block list than it is for people to switch to an entirely different provider with an entirely different namespace and try to rediscover all their friends.
If your idea is that every time a host gets blocked, everyone could move to a new one and somehow rediscover their entire social network, it would be way easier just to use a centralized service and access it with different VPNs as they are successively blocked instead. That would prevent you from having to rebuild your entire social network each time, but is still a terrible UX.
Yes but Internet still young and evolving. Here are modern solutions :
1) https://yunohost.org 2) http://cozy.io 3) https://wiki.debian.org/FreedomBox 4) http://modoboa.org
And even if everyone won't selfhost one's email or XMPP service, by owning one's domain name (DNS or GNS) he-she can switch from an hosting service to an other. And by owning one's own domain name, he-she wont loose contact with his-her social network. GNS is a good decentralize solution by separating name-identifier-locator https://gnunet.org/gns
We have to switch from old email-JID semantic user@host to the new one greeting@name and this is not a technical change https://linuxfr.org/users/apichat/journaux/salut-toto-saluta...
> If your idea is that every time a host gets blocked, everyone could move to a new one and somehow rediscover their entire social network, it would be way easier just to use a centralized service and access it with different VPNs as they are successively blocked instead.
This look like a joke. How many VPNs providers is there ? Won't be blocked ? The only thing you've done is post-pone the problem and save your centralize enclosing business.
With Signal and WhatsApp and other centralizes and enclosing social network precisely you have to rebuild your entire social network when the service go bankrupt / go wrong way or don't evolved / is compromize by intelligence agencies and law / is blocked.
Switching from Signal to WhatsAPP is very expansive because you need to rebuilt your social because you don't own your Internet name (DSN or GNS) nor a conversation address with the semantic greeting@name.
1) actuality this is not true and even if it was it could evolve. A google-centric world is not the end of history.
2) XMPP is an evolution of email and it solve many metadata and data protection problems. Conversations is a good modern solution https://conversations.im With Snowden revelations and the GAFAM domination many people do hard work to solve those problems, for example the IETF. It looks like you don't want to solve those problems because you want Signal to become a S for GAFAMS.
3) "blocking 100 or 1000 hosts is no more difficult than blocking one." Yes it is more difficult because censors have to discover all these hosts. They don't already now them. Big centralize and enclose social networks as Signal, Facebook or Gmail addresses with semantic user@host are gifts to censors and Intelligence agencies. We've saw it with Prism.
Finally is there perfect technical solutions for political problems ? No, no one is perfect, no one can solved political problems. But some of them are better than other, some make more difficult to censor and spy everybody. Big centralize solutions are the worst and are not "Internet" solutions but they are 20th century "Minitel" Telco bad solutions.
It's not and you don't, because your entire social graph is on your phone, in your address book. That's precisely the point. As the identifiers and contact lists are owned by the users and not by the messaging service providers, switching between messaging services (taking your social graph with you) is about as easy as it gets. See https://whispersystems.org/blog/contact-discovery/
I can't see how there is anything technically difficult about it. To the extent that services aren't censored, it's usually either because they're not encrypted (so no reason to block them) or because they're too popular to get away with blocking.
If all you want is for a few hundred people to be able to communicate, you don't need a federated protocol, you just need a VPN. It is much easier to use a centralized service and access it with a VPN than to use a rotating set of hosts across successively blocked federated services. At least when the VPN gets blocked you don't have to rediscover your entire social network when you start using a new one.
Federation does has the nice property that it gives you some choice over who controls the infrastructure you use, but in this case it would mean moving to a different country.
The vanilla XMPP standard doesn't permit a server to deliver the same message to more than one client for the same user. This is utterly wrongheaded and falls apart as soon as you start logging in from multiple devices (e.g. phone + computer, or even just two different computers). This was obviously terrible and has been obviously terrible for decades, but it seems impossible to change it. There is an extension to work around the problem, but extensions take forever to get approved and twice forever to get actually adopted in real clients.
The groupchat UX was always awful, partly because groupchats were never properly federated (they have to live on a single server) and there was no way to do groupchat history for people who didn't keep all their devices turned on all the time (other than, you guessed it, an extension that got stuck in limbo for a decade or so) but mostly just because all the clients sucked. It really isn't hard to do groupchats right: all you have to do is alert when the user is mentioned but not alert on ordinary messages to the group. But somehow everyone managed to screw it up.
XMPP was objectively terrible (or at least, all the XMPP clients were objectively terrible). Worse than AIM. But there were people who were (understandably) blind to its faults because it was open-source, and this anti-synergized with the traditional-unix luddite tradition that rejects anything other than plain text in messages, and the good old standards-first approach that brought us XHTML2.0 and CSS2.1 and a decade with no HTML standard (i.e. trying to standardize everything before any implementation, which never works - what works is allowing clients to add extensions and then standardizing what's been shown to work and become popular, a la WhatWG).
The real issue is that if you try to build an XMPP client that defines something "non-standard" you'll get torn to pieces by the OSS fans. Many of the niche instant messengers are probably running XMPP under the hood, but they don't want to expose it or support federation lest they get accused of trying to "embrace, extend, extinguish".
2) Wrong. RFC 6122 explicitly permits this. XMPP was designed for multiple devices, actually, but expectations changed over the past decade. Carbons (XEP-0280) does what people want now, and is widely deployed, in real clients and servers. Getting extensions to the "Experimental" state is easy, BTW - it's trickier moving them on from there because it's trickier to change them once they're widely deployed.
3) Bizarrely, you're mostly describing the client behaviour I have on every client I use. As for federation, that's what XMPP does. It doesn't do distribution, because doing that and getting autonomy for your security domain is hard (if not impossible).
4) Saying something is "objectively terrible" when it's clearly your opinion, and then rubbishing everyone who might hold a contrary one, is pathetic. The validity of your position is not helped by the XSF working on live documents for most of the time (the "Experimental" phase). Where it differs from the WhatWG is that specs move out of that phase once they're proven, and the documents become harder to change once they're Draft or Final, reflecting how hard changing all the implementations would be. The XMPP world sees people add extensions, try them out, and then move them to standards all the time. We see that from HipChat, Conversations, and plenty of others - it's a great way to work. Other folks like to get something to Experimental first, and that way get early feedback from the community. We think the way we work is a pretty good balance between stability and dynamism.
5) Absolutely not. That "X" stands for "Extensible" (I know, it was fashionable back in the '90's), and the XML namespaces you despise (and appear not to understand) mean that you can cheerfully slap whatever data you like in and alongside the standardized stuff. Trust me, the (many) IM services based on XMPP - and if it were as bad as you claim, that'd seem a poor choice - aren't refusing to federate because the OSS fans might not like it. Quite the opposite, actually.
My mistake - it is a permitted option. Still, the standard (or at least RFC 3921) encourages servers not to - delivering to all clients is mentioned as the last MAY option, and only permitted at all if they all have the same priority (which given that clients like to do things like lower priority when you haven't been typing for a while, is very hard to achieve).
> XMPP was designed for multiple devices, actually
Be that as it may, it was a lot less usable in practice than any of the alternatives when used from multiple devices.
> Carbons (XEP-0280) does what people want now
Yeah, that's the extension I was talking about. Dated 2010 but I could swear I remember it being discussed in 2008 or earlier? Looks like it may, perhaps, have finally been standardized in 2015? Of course it's still an extension so you can't rely on any given implementation supporting it, though if you're saying it's widely deployed then that's a huge improvement over what I saw a couple of years ago.
> As for federation, that's what XMPP does. It doesn't do distribution, because doing that and getting autonomy for your security domain is hard (if not impossible).
Be that as it may, it leads to very poor UX. I can have one-to-one conversations with any of my contacts as though it were a peer-to-peer messenger, but as soon as I want to have a three-way conversation we have to pick a server and a room name and .... Heaven help you if you want to add a third participant to a conversation after you've started.
RFC 3921 was superceded by RFC 6121 in 2011. You might want to update your facts before you criticize the protocol.
The message routing rules leave many things to the server implementation, adding a bit of fuzzyness, but all of this has been addressed in Carbons, and really, in the last years there was significant progress in how mobile XMPP works and feels.
as soon as I want to have a three-way conversation we have to pick a server and a room name and...
The modern mobile XMPP apps like Conversations already have a seamless 1:1 to group-chat upgrade mechanism, and people are working on writing down according specifications.
Only if you agree upfront on some kind of "safe subset" of XML.
For example, disabling entity references helps a lot in terms of reducing complexity and protecting from protentially RAM exploding XML documents.
But parsing full featured XML is hard and opens a whole can of worms.
XMPP was objectively terrible (or at least, all the XMPP clients were objectively terrible).
Do you see the contradiction?
Prosody and ejabberd exist. They don't take much hacking to make them very usable - certainly quicker than writing a protocol from scratch.
It's not that it can't be done.
Yes and no. I mean you can argue it's a matter of taste but I don't think anyone genuinely likes (for example) having to right click and go through two levels of menu, type a name, press a button, wait for results, and double-click on the correct one, just to rejoin the groupchat they were already in, every time they lose and regain connection. The line between legitimate differences in preferences and genuinely bad design is always fuzzy, but I do think there's such a thing as objectively bad UX, and every XMPP client I ever saw had it.
> Prosody and ejabberd exist. They don't take much hacking to make them very usable - certainly quicker than writing a protocol from scratch.
I can't speak to the servers. Just to the clients.
Also, people who connect Pidgin to their Gmail and think that is the pinnacle of XMPP experience, but in fact both are severely outdated.
XMPP seems a morass of optional pieces, so clients and servers implement random subsets and it makes life difficult for random users to use securely. Moxie makes a good point about and federated standard without control of client and server is going to evolve very slowly an fracture into incompatible pieces.
Signal's approach has its share of problems, too. It took OWS two years from the start of iOS development to deliver voice and text support. (They may not have been working on it the whole time, but that just underscores the problem.) The iOS app doesn't support rotation. Desktop support was a long time coming, and the multi-device experience still kind of sucks.
It's great that WhatsApp could push end-to-end encryption to a billion users at once. But none of my friends who care about end-to-end encryption trust Facebook, and Signal isn't winning over the others.
Good answers : An Objection to ‘The ecosystem is moving’ https://gultsch.de/objection.html
Juste think about how the web and Internet evolved. Juste think about how the web and Internet evolved
What percent of DNS lookups use DNSSEC?
What percent of email is e2e encrypted?
What percent of NTP lookups are signed, encrypted, or trust worthy?
What percent of HTTPS traffic would not die by a trivial downgrade attack?
What XMPP client allows you to click on it in the apple/google play store, auto discover any contacts using it, and start chatting (with e2e) in 30 seconds?
Are things getting better? Certainly, it's rather slow progress though. I don't see how signal could have gotten as far as has, as quickly as it has, without controlling both server and client, and skipping federated.
Sure, by leading by example hopefully the bar is raised. All the signal compromises (controlling client and server, non federating, using GCM, requiring a sim, etc) seem to be well justified to get pretty good security to a large number of diverse people quickly.
Sure federated would be great, as would be independence from the whisper systems servers which can be easily blocked (like in egypt).
So sure hopefully someone comes up with something that manages the strengths of signals with federation and censor resistance. Not sure that will look much like XMPP though.
The protocol in action feels crufty, and the massive number of optional extensions means that you're never quite certain if your client, your server, your correspondent's server and your correspondent's client will all play ball.
Then you toss in the fact that all the big players have decided that walled gardens are just awesome, and the fact that no-one today would touch anything using XML with a 1,024-foot pole — and thus there's just no ecosystem for it anymore.
(as an aside, I believe that in a century our children will consider the digression to XML & JavaScript to be as inexplicable as we do China's decision to scuttle its fleet of exploration)
The power and notification issues seem to be solved (at least somewhat) with some clients.
The big tech companies don't know how to make money without walled gardens (Thiel's book more or less makes this gospel without actually saying it)...hence, XMPP lost.
You can go look at XMPP or any other protocol and yeah it has issues, but no more than any other protocol, and none that couldn't be fixed.
But it's not obvious to find a working business model worth billions out of that, so companies that have to make billions don't want that. And of course government only cares about billion dollar companies (they don't have more reliable signaling), so...the fate of the protocols. There's usually a lot of bike-shedding after the fact, but they ignore the real reasons behind failure of adoption.
There's already been multiple waves of centralization, decentralization, and so on:
In the late 90s you had ICQ, MSN, AIM, etc, by the mid 2000s those had been reversed engineered to be open, new protocols were made (XMPP) and even adopted by the new giants, then Facebook, Google, etc closed back down.
As Slack is currently doing with IRC. Fun stuff.
This does not explain why small (niche) players go that route, too. It seems you overlooked the second part of the question: While I understand that big players like WhatsApp want to bind all users to their own infrastructure, I don't understand why even the niche instant messangers go through the burden of creating their own infrastructure (or relying on Google's) instead of just concentrating on the client side to provide better XMPP clients.
I don't know much about niche messengers which are not based on XMPP, but most XMPP clients happen to be niche projects themselves, maintained by one or two persons in their spare time.
The argument usually is along the lines of: "it can't work because it didn't work".
Niche players take their cues from the crowd, too. If a bunch of people in a forum say that tech is bad and that's the dominant mindset, it's going to limit the people who give it a shot.
Please see https://xkcd.com/927/
Signal and WhatsApp each provide all three: a protocol, a hosted platform and an app.
XMPP is just a protocol. There's no business model in a protocol.
No it'd be great but unfortunately the things that make something technically great don't align with the business of acquiring monetizeable users (either directly or indirectly by selling the resulting social graph).
I'm convinced that the only reason we still have email is because it predates walled garden land grabs.
Signal's creator runs their own email server, but always points out that 99% of the time, they are communicating with gmail on the other side. This subverts the decentralized nature of email.
XMPP is not a mobile friendly protocol. Even the Matrix.org folks are doing XMPP compatibility, but not implementing XMPP directly. My point is, "don't align with the business of acquiring monetizeable users" is not entirely true, and there are technical reasons (not just business reasons). Everyone with an interest in this problem should be on the same page, and holding overly cynical views about the impediments isn't (IMO) conducive to creating solutions.
We even had a piece of this for a while as Google and I think a couple other messaging providers had XMPP support but it's gradually been killed off.
The "business interests" part of this is that anybody creating a monetizeable user base via a messaging network wouldn't give the opening to others to come in via XMPP. It gives your users a way out and gives your competitors a way in.
http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...
| The next few years were spent re-writing and modifying quite a few parts of ejabberd, including switching from XMPP to internally developed protocol, restructuring the code base and redesigning some core components, and making lots of important modifications to Erlang VM to optimize server performance.
I recently tried XMPP on Pidgin for the first time on my laptop (Ubuntu), and I hate it. There's something very wrong and very ugly about the UI. It's feels just wrong. And not just that, I had to jump thru numerous hoops before I could get the thing working.
I also recently tried to install Signal on my Xiaomi Redmi2, and it doesn't work. I don't know why but the registration always fails when "connecting to the servers", even though everything else on my phone works. (before I start getting responses about this, if your suggestion can be found on Google I already tried it)
I think some things, like messaging where it's easy to create an MVP but hard to get the number of users that actually brings in the value, the entire game is polishing your one client to reduce frictions and increase experience. WhatsApp works as flawlessly on mi Redmi2 as it does on any Samsung or any iPhone. When you have zero friction to join over time your user base grows. When you have enormous friction to join, where users have to be wrestling with your broken app, it just fizzles. This isn't the case for every industry, but seems to be the case for messaging. Another example is Telegram. At one point they were ahead of WhatsApp in terms of features, now they're behind. Regardless, they keep growing their user base. My opinion is this is because their UX is at least as good as WhatsApp and has just as little friction.
Not on Google maybe, but still the page that you want: https://github.com/WhisperSystems/Signal-Android/issues/new
But ...
What exactly happened that XMPP lost on mobile phones? What did they do wrong? XMPP was there before smartphones came, had working clients, and had (for that time) pretty decent encryption.
While I understand that big players like WhatsApp want to bind all users to their own infrastructure, I don't understand why even the niche instant messangers go through the burden of creating their own infrastructure (or relying on Google's) instead of just concentrating on the client side to provide better XMPP clients.
Is XMPP so bad that nobody wants to do that? If so, why?