This is exactly the problem that XMPP is designed to solve. You pick an XMPP provider who you trust, and you can get cross-provider traffic through XMPP Federation. One of the largest XMPP providers is Google Talk, which lets you hook right into the standard GChat.

The problem is precisely that people keep re-inventing the wheel here. We don't need Facebook Chat and all the other deliberately-incompatible protocols that keep getting built out, we just need people to implement XMPP.

Having looked at this in some detail, my feeling is that no federated solution can work. The reason is that the underlying architecture leaks through to the user experience in unacceptable ways.

As soon as a user has to understand federation - at all - I believe the proposition is dead to most people. For example Diaspora's 'pods' are, I think, fatally confusing. (I think Zuckerberg realised this, which is why he donated money to Diaspora - a project he could see would fail, but might also crowd out other OS alternatives to Facebook.)

For sure, XMPP can be part of the tech solution, in fact I suspect there are really excellent open source tools to address many of the inherent issues (perhaps Apache Cassandra can be Twitter's contribution to democracy) but I think the real problem is a social one.

You need an institution that can credibly promise to provide a platform in the long run and which can also get critical mass in at least one community. I believe the critical mass might well follow if the institution is widely perceived as reliable and legitimate in the way a foundation like Wikimedia is.

I went into some more detail on this in a blogpost: http://jimmytidey.co.uk/blog/building-a-better-twitter/

> no federated solution can work

I don't see why not. Is the problem fundamentally different than SMS messaging or email?

Two thoughts: - How would you provide the functionality that hashtags do on the public timeline?

- How would users know which nodes they could trust?

I think email and Sms are pretty weird user experiences that have succeeded in part because there was no alternative.

> How would you provide the functionality that hashtags do on the public timeline?

That is what search engines are for. The whole point of the Internet is that you can build businesses on top of open protocols with interoperable implementations. Giving up on open protocols and control of data (you can't even save images on Instagram...) "because hashtags" is incredibly shortsighted.

Twitter is already a federation of servers and services. It doesn't seem outrageous to imagine another server or cluster of servers being owned by a second party.

I think we may be talking past each other here, what I mean is that federation must be strictly and absolutely invisible to users.

Perhaps I was wrong to say that federation is in principle ruled out, but I do think that it must be invisible to users. - Including namespacing usernames to nodes within the federation as with the @ sign in email.

I believe this is true in spite of the technical beauty of federated solutions, and it's exactly that beauty that's lured so many onto the rocks.

I don't think the namespace issue is as big of a deal as you seem to think it is. People are used to dealing with namespaces when they make phone calls, send emails, send a letter, get a license plate, or go to a website.

If it's not federated, it's not a solution. As for that being "too confusing" to people, well, you can't save everybody.

> we just need people to implement XMPP

Isn't the meme that XMPP is horribly complex and the support for various sub-standards varies hugely?

Reality. That is the reality.

> We don't need Facebook Chat and all the other deliberately-incompatible protocols that keep getting built out, we just need people to implement XMPP

Which is funny because Facebook Chat also used to be powered by XMPP

I doubt it, they had an XMPP interface, but that doesn't mean they use XMPP in the backend

I'm not sure I fully understand this comment. What does it mean to have an "XMPP backend" but not as the interface? And yes, they did use a full, open source XMPP solution for Facebook chat when it started out called "ejabberd". Ejabberd is a terrific solution if you quickly want to have a scalable, real-time messaging and event solution with low maintenance.

I'm not the person you responded to but I would assume that it would look like any other service that uses a different protocol or format on the front-end than the back-end. E.g. a REST service that talks JSON on the front end but speaks SQL to a database. So they have an XMPP backend that uses a translation layer to send a custom AJAX or socket-based protocol to the browser.

Yeah, I was pretty sure that Facebook was just using straight-up XMPP at one point in time. They have been actively trying to drive traffic to their Facebook Messenger app - first by shutting down their XMPP interface, then blocking applications that talk to their web interface, and finally by removing Facebook Messenger functionality from their mobile website.

Partially. Part of the XMPP protocol is establishing presence which I don't think is a requirement for Twitter (I wouldn't know, I don't use Twitter). Establishing presence for XMPP usually means a constantly connected client via TCP. The protocol would probably have to be tweaked so that clients don't necessarily have to remain connected to use the service. Might be possible to do this in XMPP but would require some work.

Are XMPP messages published for public viewing? That's the use case of Twitter, not private chat.

XMPP is simply a publisher/subscriber architecture. So you can just have a "global" publisher that any user subscribes to when they get on the service (registered user or not) for public viewing. The protocol isn't limited to just private chat.

So then XMPP doesn't solve this problem as you need a service which saves and publishes the message for public viewing.

I'm in agreement that I wouldn't necessarily choose XMPP for Twitter but under different reasons. There is literally nothing holding back the XMPP protocol from saving and publishing messages to the public viewing. XMPP is the eXtensible Messaging and Presence Protocol and suppose to be extremely modular so I really don't think what you're describing is impossible.

How do you propose to prevent spam and abuse of this open federated chat network?

Realistically it would have to work the same way as spam filtering does for email.

Come to think of it, it's surprising that spamming is not more prevalent on SMS networks, because it's the same kind of federated system. But I guess access may be gated well enough and enforcement strong enough to prevent that.

SMS spam is common in Europe.

And Hangouts is only partially interoperable with XMPP.

Maybe it was too early back then to see the value in it, but does anyone remember App.net? Dalton Caldwell (now working at YCombinator) had a plan to build a backbone for developers to build on top off without fear of APIs being taken away from them[0]. The idea came about from this exact topic (Twitter becoming more and more hostile to developers)[1].

[0]http://daltoncaldwell.com/an-audacious-proposal [1]http://daltoncaldwell.com/what-twitter-could-have-been

I remember App.net with fondness. I was a paying customer and the community was awesome, pretty much entirely troll-free techie types, and I never felt awkward just jumping into a random thread or conversation.

First was the great apps, and the features that continued to roll out for users. Unfortunately it was under appreciated.

App.net proved that locking down the API wasn't Twitter's problem. Once people got over the initial phase of trying out every App.net app they stopped using it. It was the thing that brought them there, but it wasn't enough to keep them using it, regularly.

What happened to it?

Income < expenses.

Same as Twitter then

Were the APIs taken away?

"The bad news is that the renewal rate was not high enough for us to have sufficient budget for full-time employees. After carefully considering a few different options, we are making the difficult decision to no longer employ any salaried employees, including founders." http://blog.app.net/

So effectively development stopped in 2014 and it's on maintenance mode since then.

A large mesh of zeromq could do it, and let the clients deal with storage.

Twitter has very limited use. It's good for news/media companies to announce headlines and perhaps for businesses to reach customers with product announcements or recalls.

But in terms of having any sort of meaningful discussion on Twitter or the sharing of anything approaching deep thought or analysis, forget it. To me, there is close to nothing of value that can be added by the ordinary individual except perhaps links to other sites.

Personally, I despise Twitter. I do not use it. Never had an account, and never will. I don't know what the future is for the company and the service, but I can only hope that if they are purchased the acquirer does it purely to remove competition from the market and has a celebratory dumpster fire with the servers. If only I was so lucky. Twitter is part of the ego machine that feeds the narcissists that pervade social media.