I worked at Google at the time. It was killed for engineering reasons, that boiled down to:
1. Nobody used federation.
2. Except spammers. They used it a lot. Trying to keep federation alive whilst fighting spammers took a lot of effort.
3. It complicated the code a lot. Features that existed in GChat but didn't map well to XMPP were harder to implement.
4. The XMPP protocol sucked on mobile and sucked in web clients, and therefore Google's own clients were not using it any more. New extensions like Jingle were more like entirely separate protocols than small upgrades.
Federated chat protocols are the sort of thing that intuitively sound nice, but networks that implement them inevitably end up being killed off by closed, proprietary competitors that are simply a whole lot better. Email hangs on, kinda, but I passively await the day that the email system is killed off by a closed network. It already happened for personal correspondence (facebook) and I'm sure at some point it'll happen for work correspondence too.
Moxie Marlinspike has written some thoughts on why federated protocols are yesterday's solution here:
https://whispersystems.org/blog/the-ecosystem-is-moving/
Put simply, in a world where clients are all free and the identifier of choice is the phone number, federation doesn't add much value.
To me this translates to:
“Please run our proprietary software to participate and use a high-value nearly unique identifier to identify yourself — we really like to know that [email protected] is the same person as feetfetish33 so we can better quantify you for targeted advertising.”
I am willing to entertain the notion that federation is not a solution, but seeing how you now either place yourself in this panopticon or you don't and 'miss out' on things makes me feel that we should at least attempt to figure out a way to put communication back in the hands of the people.
For all its warts and issues, email is still effectively federated, and will stay like that for quite a while due to the needs of many (not all, I certainly grant you that) businesses and local, regional, and national governments to control their own email infrastructure (mainly due to legislation and control).
Chat networks are all moving to the use of phone numbers because users mobile phonebooks are a vendor neutral, open access social network of high value contacts that almost everyone has and for which there are simple APIs available.
Phone numbers have other advantages. They are difficult to register in bulk (it can be done but it costs a lot more than bulk registering web accounts protected only by a CAPTCHA). Regulations in many part of the world enforce the ability to do number porting which makes mobile numbers truly user owned - unlike email or jabber ids which are ultimately owned by the organisation after the @ symbol. There is a simple remote attestation protocol: you can prove you own the identifier by simply providing a challenge code. Everyone understands them. And it outsources identity management costs to the telcos who have large branch/shop networks to help people who e.g. lose their device/SIM. Building out and staffing account recovery infrastructures is a significant driver of costs for large web platforms. For instance if you have a contract then you can recover your identity by physically walking into a local telco shop with your passport, you will walk out with a replacement SIM (and the prior SIM remotely deactivated) a few minuets later. It's partly by shifting these costs to the telco networks that WhatsApp was able to scale to hundreds of millions of users with only 50 employees.
When I look at how things work done this way vs a traditional internet federated network like email, Jabber, IRC etc, I have to agree with Moxie - it's not so bad, actually. I'm not normally a big supporter of government regulations, but making number porting obligatory is a relatively low cost rule that makes the use of phone numbers as the universal id a lot more palatable, because it's truly user owned at that point. Switching mobile networks and switching chat networks is a lot easier than switching email/xmpp providers because forwarding has always been an afterthought in such protocols, is legally optional, and at any rate is always going to be more complicated than just re-assigning ownership of a truly provider independent code.
It's got a few years yet, but POTS and mobile are already legacy, they just don't know it yet.
Better hope Apple shareholders don't find out!
It's not the _devices_, per se. It's the network. The infrastructure. And the ability for Google to rely on a phone number per person. It's an invalid model.
And voice comms are occasionally useful, though I make them rarely -- it could easily be months.
The idea of carrying a bundle of angry that can start sounding at any time, anywhere, is a turn-off. Especially if I've no control of who's at the other end.
Two of the primary reasons people carry phones are because others demand that they be reachable, and secondarily, so that the phone holder can reach others. The second I don't mind as much though it's also a crutch.
A device with good text capabilities, that can also optionally receive voice inputs and convert that to text, allows a very* limited whitelist set of calls, and otherwis directs all incoming traffic to a wait or prove your worth queue, would start to approach reason.
I remember the days of five-line dial phones and office receptionists, pre voicemail. It sucked for the receptionist, but coming back or into the office and being handed a stack of message slips was vastly preferable to bouncing through voicemail and having to do the transcriptions yourself.
You won't see that because that is not what makes the phone number so useful. Its biggest benefit is being able to correlate all those separate data profiles via one unique key. This makes these profiles more valuable to advertisers. You won't see advertisements in most of these apps (it would drive users to the competition) but they don't have to; they just show them in the browser. They already know it is you thanks to the ubiquitous social media buttons and tracking going on.
This is largely conjecture, sure, but the fact that using your phone number is a requirement rather than an option makes me suspicious.
Sometimes I don't mind if two services know that I am the same person, but I would like to be able to choose for myself. Sometimes I do like to use a throw-away email address and a VPN to use some service, simply because I care about my privacy. Often I don't feel that a service needs to know who I am beyond an alias.
What to do if people (majority) don't care ?
Because that's the side result of bringing uneducated (regarding IT) masses online. They sell on instant gratification, cute shit (emojis, gifs galore) and hype. In return they don't hesitate to give their data and conversations.
No wonder everyone want's their own silo.
I don't know the numbers but I'm part of the population that never used Facebook, so I'm unconvinced you can generalize like that.
> Put simply, in a world where clients are all free and the identifier of choice is the phone number, federation doesn't add much value.
I didn't really understand Moxie's arguments, but he may know more due to the projects he's involved in.
But how is interoperability achieved? The beauty of HTTP, NNTP and SMTP are that you're not bound to one (or maybe two) client experiences or server implementations. I don't understand the desire to leave achievements like that on the floor just for temporary comfort. It's a slippery slope and will lead us into a long time of silo'd communication.
To me the current proposals look like if I had to use 4 different post offices depending on whom I want to send a package to. There's a reason addresses are universal.
If you want to make a better WhatsApp, all you have to do is ... build a better app. The traditional obstacles are largely gone in the mobile world:
• You can access the same social graph just by requesting the contacts permission on your mobile phone: it's not like Facebook where the social network is locked away.
• You don't have to go through some slow moving standards process which takes years to get the features into the most widely used clients like you would if trying to improve XMPP.
• There's no cost to the user having multiple messaging apps installed thanks to Google/Apple's push networks: it's not like Windows/MacOS/Linux where having an app running in the background uses resources and requires permanent on-screen reminders of the resource wastage. So it's easy to get the user to set up your app.
• Notifications will appear in the same place the user is looking for them no matter what.
• You can (on Android) register for the right intents and whenever a friends phone number is invoked, your app will be offered as an option, so there's no lockin there either.
These things together mean the traditional reasons for pursuing federated networks for chat (the avoidance of lockin) are largely irrelevant.
I'm looking to wearables and low-power IoT devices, and for those we may need a new federated identity/discovery scheme. The DNS is practically ancient now, but has served email, web and XMPP and many other protocols besides surprisingly well for decades. I doubt it is sophisticated enough for global-scale wearables and ubiquitous sensors. We could never have built anything so sophisticated on phone numbers.
Most of your remarks seem to align with the preferences of chat app developers. But actually there's no group I'd trust less, today, to determine the future trajectory of communication. Interoperability, federation and end-to-end behaviour is the open architecture of the Internet, and I believe anything that undermines that triad should be met with contempt and resistance.
Re sharing contacts by replacing User@Domain with numbers:
Yes, this solves some of it because everyone agreed to use phone numbers, but phone numbers are a thing of the past and you don't always want to share your phone number just for messaging.
Re notification overhead:
If you have a system-wide notification display system, then of course the resources it requires are limited but you still have to have it running in the background in some form. Granted, it's probably easier to optimize it with everyone using the same notification system. This is just consolidation of a popular feature into system-provided functionality.
Re lockin:
But lockin still exists because you are the whim of centrally managed for-profit messaging backends, operated by entities whose intentions may not necessarily align with yours.
I'm not aware of cross-app synchronization of messages and even less so a common message format (feature) set supported by all that provides a rich experience.
I don't disagree with the cited shortcomings of XMPP, though you can always find a flaw in something which wasn't explicitly designed for the current use case, so ignoring that, the basic premise of XMPP is still sound and needed. Implementation details are something else, and honestly I don't agree that replacing XML with JSON (as in some of the proposals) gains anything in terms of efficiency, which it probably wasn't intended to anyway.
My impression is that building a messaging system is simple enough that many variants pop pup, but almost all of them get interoperability, synchronization, mobility, and security wrong. It's unsurprising because the simplicity attracts implementers of all domain experience levels.
lockin still exists because you are the whim of centrally managed for-profit messaging backends
Given that 99.99% of people will never run their own servers, this is irrelevant: someone will be managing the infrastructure and those people will be motivated by profit. Welcome to capitalism: it works better than the alternatives despite its flaws.
Things like WhatsApp or GChat do not simply replace XML with JSON. They tend to use binary protocols designed for low power consumption and ease of parsing. Arguably, if any protocol gets it wrong, it's XMPP ... I used to like it back when it was called Jabber, heck, I even hung out with a few Jabber developers back in the day. But Jabber's design goal was instant messaging and it's no longer useful for that. It just couldn't adapt to even quite small changes in circumstances.
I didn't say there has to be. What I said is there's a single notification service which is reused by everyone. If you're saying that there's no daemon of sorts for notifications, then I'm curious where it's implemented.
> Given 99.99% of people will never run their own servers, this is irrelevant
It doesn't matter that someone won't run their own server. What matters is that you can, which means someone you know or trust can and give you access. These days it's much, much easier to run simple services like messaging if you consider the proliferation of remote accessible NAS boxes and such in households. Adding a messaging service to that is easy. Other than that you can use Sandstorm too if you don't want to operate a server.
> Things like WhatsApp or GChat do not simply replace XML with JSON
I'm sorry you thought I said WhatsApp or GChat use JSON. I mentioned it in reference to other messaging services.
If I want to chat with person X, I either have to use the same client as them, or I can’t do it.
I can’t write my own, better client, either.
That’s the big issue.
But then again in Europe iPhone market share is much lower.
When you removed XMPP and forced to hangout it died off pretty fast, no API and the Hangout thingy was not even working on Linux (not sure if it does these days, i never tried).
I see your points, but i cant stop to think about this as the probably most evil step google did to my workflow.
Edit:// I use hangout now to transport links from phone to desktop and the other way around. It totally lost its purpose :/
I think mobile chat taught Google a lesson: the cultural Dna that "open always wins" pretty much leads to losing and having to play catch up.
I think at this point the biggest hope is that something like telegram can become the defacto new xmpp.
Why would gchat have as many users as facebook anyway? It's not like gmail was a community like facebook was. MSN meanwhile basically started as a chat client, there where ads for it on tv in Sweden...
Since a critical feature of a messaging app is "my friends are on it", that gave MSN/FB/AIM a huge advantage over GChat, and had they not turned off XMPP integration, they would have bled off users until it was only used by small pockets of people, all of whom were on GChat (eg. Googlers).
How this had worked?
I mean, in XMPP both parties need a JID, so MSN users must have had theirs.
Had MSN had generated random non-discoverable JIDs for their users, blocked any incoming messages unless there were prior outgoing communications, made GChat interop opt-in or what?
I think maybe the grandparent was saying that you couldn't add msn friends to your contact in gtalk, but you could add gtalk friends to your msn. Or maybe its something to do with emoticons?
Maybe this high-lights the drawbacks of being too focused on your core business, so focused that you cut of an arm (gchat) because it wasn't in line with your philosophy of having a totally open, standardized protocol that you (Google) hoped everyone eventually would be using. The players of today do not care about these things, they care about getting traction and a massive user-base and about encryption and privacy and claim their place in the spot-light because users love them.
Google knew we loved gchat. They must have know why. Google Drive + gmail + gchat, tailored towards businesses: good bye Office 356, see you never. Microsoft would be limping without Office... how could Google fail to make this happen?
It also didn't make any sense they dropped XMPP in favour of Hangouts. XMPP has its share of problems, but it's designed as an extensible protocol. Google's Jingle addon to XMPP was actually quite nice, although it took quite some time to get polished.
Once they had it right and it was gaining some traction, they drop it in favour of a proprietary solution...
It makes perfect sense. Forcing out competitors (XMPP) with leverage from dominance in a neighboring market (gmail, which was the common interface for GTalk) is a textbook example of monopolization.
If we lived in a world that actually cared about the rule of law, Google would be charged for violating the Sherman Antitrust Act (and/or related antitrust acts).
Not to mention that XMPP is a standard, not a competitor.
This is like saying that modern companies are monopolizing against SOAP or YAML.
I didn't say anything about other players in the "chat" market.
> Google at some point monopolized the "instant messenger"
No, the have dominance in the email market (I mentioned gmail). They used the position of power in the email market to limit XMPP and support their new Hangouts service.
Just like Microsoft used to do, Google broke their support of the protocol. While most of the protocol still worked, they started dropping auth requests on the floor. This wasn't an error message or missing feature; their S2S protocol simply dropped auth requests so it looked from the outside that the request was delivered, but the person on the other end never responded. Outgoing (from gmail) auth requests were removed entirely. From the perspective of gmail users, XMPP popularity simply faded and Hangouts took over as a replacement.
Using one market (gmail) as leverage in another market (chat) is basically the definition of monopolization. Note that this doesn't require a perfect monopoly in the first market; ability to force/manipulate the market is sufficient. Additionally, the Sherman Antitrust Act also criminalizes the attempt to monopolize.
> Not to mention that XMPP is a standard
Obviously. I've been following the development from the beginning (before it was called "jabber"; the name "XMPP" happened many years later).
> not a competitor.
An open (federated) protocol is the most dangerous kind of competitor to a big company like Google. Deals/contracts can can be made with a company, but a federated protocol by definition cannot be controlled by a single entity.
> This is like saying that modern companies are monopolizing against SOAP or YAML.
I think you need to re-read what I said. Or maybe read more about how antitrust law works.
No its not. Unless you're claiming that this is a form of product tying, but that would require that
1. Gmail held a monopoly or near monopoly over the email market at the time 2. hangouts was a service that people needed (or that to use gmail, you also were forced to use hangouts)
Neither of those were the case.
There's no such thing, that's a strawman to permit calling things monopolies when they aren't. Given that Google allegedly tried to act like a monopoly and totally failed, I suggest that you rethink your view.
> An open (federated) protocol is the most dangerous kind of competitor to a big company like Google.
You mean protocols like SMTP and HTTPS, protocols which are core to Google's business? This was not a company with a monopoly in one area leveraging it to gain advantage in another. Google was not even close to a monopoly (or even a plurality) in email; Yahoo for one had more users at the time than Gmail.
This was Google admitting that open wasn't working and going private to try and retain existing users. Which, not having anything close to a monopoly, was perfectly legal. One could even argue it was their fiduciary responsibility to their shareholders to try and grow the user base.
Historically, Google drops features when they don't make financial sense or when they give leverage/advantage to competitors.
But I think the termination of Google Wave (which was also federated and built with XMPP) was the real moment Google gave up on team communications.