Having looked at this in some detail, my feeling is that no federated solution can work. The reason is that the underlying architecture leaks through to the user experience in unacceptable ways.

As soon as a user has to understand federation - at all - I believe the proposition is dead to most people. For example Diaspora's 'pods' are, I think, fatally confusing. (I think Zuckerberg realised this, which is why he donated money to Diaspora - a project he could see would fail, but might also crowd out other OS alternatives to Facebook.)

For sure, XMPP can be part of the tech solution, in fact I suspect there are really excellent open source tools to address many of the inherent issues (perhaps Apache Cassandra can be Twitter's contribution to democracy) but I think the real problem is a social one.

You need an institution that can credibly promise to provide a platform in the long run and which can also get critical mass in at least one community. I believe the critical mass might well follow if the institution is widely perceived as reliable and legitimate in the way a foundation like Wikimedia is.

I went into some more detail on this in a blogpost: http://jimmytidey.co.uk/blog/building-a-better-twitter/

> no federated solution can work

I don't see why not. Is the problem fundamentally different than SMS messaging or email?

Two thoughts: - How would you provide the functionality that hashtags do on the public timeline?

- How would users know which nodes they could trust?

I think email and Sms are pretty weird user experiences that have succeeded in part because there was no alternative.

> How would you provide the functionality that hashtags do on the public timeline?

That is what search engines are for. The whole point of the Internet is that you can build businesses on top of open protocols with interoperable implementations. Giving up on open protocols and control of data (you can't even save images on Instagram...) "because hashtags" is incredibly shortsighted.

Twitter is already a federation of servers and services. It doesn't seem outrageous to imagine another server or cluster of servers being owned by a second party.

I think we may be talking past each other here, what I mean is that federation must be strictly and absolutely invisible to users.

Perhaps I was wrong to say that federation is in principle ruled out, but I do think that it must be invisible to users. - Including namespacing usernames to nodes within the federation as with the @ sign in email.

I believe this is true in spite of the technical beauty of federated solutions, and it's exactly that beauty that's lured so many onto the rocks.

I don't think the namespace issue is as big of a deal as you seem to think it is. People are used to dealing with namespaces when they make phone calls, send emails, send a letter, get a license plate, or go to a website.

If it's not federated, it's not a solution. As for that being "too confusing" to people, well, you can't save everybody.

> we just need people to implement XMPP

Isn't the meme that XMPP is horribly complex and the support for various sub-standards varies hugely?

Reality. That is the reality.

> We don't need Facebook Chat and all the other deliberately-incompatible protocols that keep getting built out, we just need people to implement XMPP

Which is funny because Facebook Chat also used to be powered by XMPP

I doubt it, they had an XMPP interface, but that doesn't mean they use XMPP in the backend

I'm not sure I fully understand this comment. What does it mean to have an "XMPP backend" but not as the interface? And yes, they did use a full, open source XMPP solution for Facebook chat when it started out called "ejabberd". Ejabberd is a terrific solution if you quickly want to have a scalable, real-time messaging and event solution with low maintenance.

I'm not the person you responded to but I would assume that it would look like any other service that uses a different protocol or format on the front-end than the back-end. E.g. a REST service that talks JSON on the front end but speaks SQL to a database. So they have an XMPP backend that uses a translation layer to send a custom AJAX or socket-based protocol to the browser.

Yeah, I was pretty sure that Facebook was just using straight-up XMPP at one point in time. They have been actively trying to drive traffic to their Facebook Messenger app - first by shutting down their XMPP interface, then blocking applications that talk to their web interface, and finally by removing Facebook Messenger functionality from their mobile website.

Partially. Part of the XMPP protocol is establishing presence which I don't think is a requirement for Twitter (I wouldn't know, I don't use Twitter). Establishing presence for XMPP usually means a constantly connected client via TCP. The protocol would probably have to be tweaked so that clients don't necessarily have to remain connected to use the service. Might be possible to do this in XMPP but would require some work.

Are XMPP messages published for public viewing? That's the use case of Twitter, not private chat.

XMPP is simply a publisher/subscriber architecture. So you can just have a "global" publisher that any user subscribes to when they get on the service (registered user or not) for public viewing. The protocol isn't limited to just private chat.

So then XMPP doesn't solve this problem as you need a service which saves and publishes the message for public viewing.

I'm in agreement that I wouldn't necessarily choose XMPP for Twitter but under different reasons. There is literally nothing holding back the XMPP protocol from saving and publishing messages to the public viewing. XMPP is the eXtensible Messaging and Presence Protocol and suppose to be extremely modular so I really don't think what you're describing is impossible.

How do you propose to prevent spam and abuse of this open federated chat network?

Realistically it would have to work the same way as spam filtering does for email.

Come to think of it, it's surprising that spamming is not more prevalent on SMS networks, because it's the same kind of federated system. But I guess access may be gated well enough and enforcement strong enough to prevent that.

SMS spam is common in Europe.

And Hangouts is only partially interoperable with XMPP.