News for hackers. Stuff that matters.
Re:XMPP, a comeback story by oofbey on Wednesday November 17, 2021 @10:22PM (#29261653)
I worked on a large XMPP system for a while and realized it's actually not a very good protocol, for one reason: presence. "Presence" is the feature that shows a little indicator next to each one of your contacts indicating whether they're online or away.

The problem with presence is that it generates a LOT of traffic. It's N^2 with the average number of contacts each user has - every time a user changes presence, you have to send a message to every one of their contacts to update them about the presence. This happens whether they're paying attention to your client or not. So on a large system, the big majority of the traffic is just presence messages flying back and forth saying who is at their keyboard and who isn't. This is really wasteful in general. Worse, it's not a super important feature for a lot of people, so it's definitely not worth the effort. And ever since mobile computing became ubiquitous, it's not even clear what this is supposed to represent.

So, I say let XMPP die. It was a fine idea for its time. But the protocol has no place in the modern world.

Re:XMPP, a comeback story by f1refly on Thursday November 18, 2021 @04:45AM (#29263767)
Actually, it is an important feature for a lot of people, with virtually every popular messenger implementing it. Telegram and Whatsapp both go one step further and not only show when a user is at his keyboard, but also if he's currently typing. In the grand scheme of things, a few strings being routed regarding presence information don't pose a big problem for server providers (I know two of those and they didn't complain nor think of it as an issue whatsoever).

You shouldn't impose your use case to "a lot of people", and more importantly also shouldn't conclude that it's "definitely not worth it" from that. The industry disagrees with you, and so does my mom.

Re:XMPP, a comeback story by oofbey on Thursday November 18, 2021 @09:50AM (#29266019)
When somebody is typing or not does not get broadcast to every contact. It’s only sent to the individual who is about to receive the typed message. O(1) communication vs O(N). That is a different feature which is not XMPP presence and I agree way more relevant in the modern world.

Even if the server load from presence is acceptable (we clearly disagree here) it’s not okay to bombard somebody’s mobile device with these messages all the time. If your phone’s CPU and radio need to wake from sleep every time any one of your contacts picks up or puts down their phone, your battery life is ruined. You may think these messages important, but they are not urgent.

Re:XMPP, a comeback story by f1refly on Thursday November 18, 2021 @03:00PM (#29269706)
Honestly, I can't disagree with you on technical terms, because what you're saying makes sense and on other topics this discussion would be over.

But in this case, I have Conversations on my phone and can say from experience that it didn't degrade my battery life whatsoever. I know that android restricts this behaviour heavily by default and for good reason, because lots of apps will drain the battery in a matter of hours. No idea how the conversations devs do it, there must be a quirk to it I don't know which makes modem usage acceptable. I'm not a zealot in this case and I accept the flaws, but I can't leave you thinking mobile xmpp hurts the battery in a relavant way because that's (in my experience) not true.

I'm not deep into xmpp xep standards, but I'd imagine theres a way to tell the server "I don't care about event type x please don't send those until I ask for them again"

Re:XMPP, a comeback story by Flowdalic on Thursday November 18, 2021 @05:59AM (#29264125)
I wonder why people jump fast to conclusion that to "let XMPP die", when the protocol can also be iteratively improved. Presence is not required in XMPP, its an optional feature. Everything you said has been considered in newer XMPP extension protocols (like XEP-0396: MIX).
Re:XMPP, a comeback story by oofbey on Thursday November 18, 2021 @10:03AM (#29266189)
Why improve when you can start over? Modern tools for pub sub and websockets are so much better. We don’t need to use XML any more. (For the love of god please let XML die.) The old code and implementations are more of a liability than an asset, especially if you want to take out major functionality. Unless interop with AOL instant messenger’s install base is a key feature or something. (Don’t know if AIM uses XMPP but the timing is about right and representative of what you’d get.)
Re:XMPP, a comeback story by Zash on Thursday November 18, 2021 @10:25AM (#29266470)
Why start over when you can improve? You'll most likely end up with the second system effect, probably slowly rediscover all the valid reasons for the things you disliked, probably re-make all the mistakes.
Re:XMPP, a comeback story by detaro on Wednesday November 17, 2021 @10:26PM (#29261677)
That's afaik a problem with pretty much any system that supports presence, even though it can be somewhat optimized potentially (where I don't know how many options exist in XMPP, but I don't think would be fundamentally impossible to add)?
Re:XMPP, a comeback story by fshbbdssbbgdd on Wednesday November 17, 2021 @10:32PM (#29261701)
It seems like it would be much more efficient for the presence to be stored somewhere where clients can fetch it if needed.
Re:XMPP, a comeback story by oofbey on Wednesday November 17, 2021 @10:41PM (#29261745)
Yeah, any kind of aggregation would help. But XMPP dictates that you "MUST" broadcast presence messages to all subscribers every time it changes. (Many open standards are very forceful in their language about such things.)

Most chat systems have done away with it. Slack has it, but that's about it. For anything you use from your phone, what does it even mean?

Re:XMPP, a comeback story by georgyo on Thursday November 18, 2021 @12:07AM (#29262153)
> Most chat systems have done away with it. Slack has it, but that's about it.

Slack, Discord, Matrix, FB Messenger, Instagram all have it to name a few.

The only ones that don't have it are SMS replacement apps (Whatsapp, Signal) and IRC.

I for one greatly rely on presence. It very often informs if I should or should not send a message to someone.

Re:XMPP, a comeback story by mulmen on Thursday November 18, 2021 @12:33AM (#29262279)
> I for one greatly rely on presence. It very often informs if I should or should not send a message to someone.

Do you have an example? I certainly don’t operate this way. I just fire off a message. The status only informs my expectation of when a reply might arrive.

In the absence of a status I have no expectation of a reply. With mobile devices I still get immediate replies frequently because of notifications. So I don’t rely on status.

Re:XMPP, a comeback story by GoblinSlayer on Thursday November 18, 2021 @03:57AM (#29263513)
In my experience away status is uninformative. It's often inaccurate - transitions too early, you can also send the message in the gap between afk and status transition. On the other hand when I'm at the keyboard, it doesn't mean I'm free for chat and can easily reply 10 minutes after I receive the message.
Re:XMPP, a comeback story by detaro on Thursday November 18, 2021 @12:07AM (#29262162)
IRC has away-status at least.
Re:XMPP, a comeback story by namdnay on Thursday November 18, 2021 @05:03AM (#29263860)
+ Teams, Jabber, Sametime, Hipchat, FB workplace

i.e. basically every corporate chat system

Re:XMPP, a comeback story by grishka on Thursday November 18, 2021 @02:08AM (#29262859)
Huh? 100% of instant messaging apps I use daily have presence. Telegram has it, VKontakte also has it, Discord has it. Not sure about Matrix, but I open it maybe once a month.
Re:XMPP, a comeback story by detaro on Thursday November 18, 2021 @02:18AM (#29262915)
Matrix has it, but it's e.g. disabled on the matrix.org instance because of performance issues.
Re:XMPP, a comeback story by grishka on Thursday November 18, 2021 @02:41AM (#29263051)
Ah ok. I'm indeed on matrix.org, I did check if there's presence before posting that comment, and didn't see any.

I do keep wondering though what's the deal with the performance of the Matrix protocol. From reading the overview in the spec, it doesn't feel like there's anything that could have a significant performance cost. It's essentially a standardized but extensible REST API that passes JSON objects over HTTP. Is it that the existing server implementations aren't optimized very well?

Re:XMPP, a comeback story by Arathorn on Thursday November 18, 2021 @03:11AM (#29263255)
It used to be that Synapse (the server) hadn’t been optimised much, but we have been going through doing so. Presence just hasn’t come to the top of the list yet as we think there are more important features to prioritise.

The reason there is stuff to optimise is that Matrix passes JSON around over HTTP in a byzantine fault tolerant manner: every time you send an event you have to sign it and justify why you’re allowed to send it, and every time you receive one you need to execute that proof effectively and do a merge resolution algorithm to authenticate the event and store it in the right context - very similar to how Git works, but with events rather than commits. A naive implementation of this can be very heavy. We’ve improved it by 3-4x over the last year or so, and there are other big algorithmic improvements in general on the horizon too.

Re:XMPP, a comeback story by GoblinSlayer on Thursday November 18, 2021 @03:41AM (#29263406)
My client has status settings that allows to disable automatic away status, so online-offline is the only status change unless you manually set away status.
Re:XMPP, a comeback story by detaro on Wednesday November 17, 2021 @10:42PM (#29261757)
oh, it's a core thing and not open for extensions changing it or being disabled? That's indeed unfortunate.
Re:XMPP, a comeback story by oofbey on Wednesday November 17, 2021 @11:04PM (#29261856)
Core indeed. The first P in XMPP standards for Presence.
Re:XMPP, a comeback story by detaro on Wednesday November 17, 2021 @10:39PM (#29261735)
Fetch has the downside that with the usual idea of presence you want to know immediately if presence changes, so you'd be polling constantly. What could make some sense is caching server-side and allowing the subscription to be quickly enabled/disabled, so that a client could save traffic while it's not on screen etc. When it returns, it asks for the last message for each contact to be delivered from cache and the subscription to be resumed (in exchange for having to wait for the data to come in, and until then not showing the correct information).
Re:XMPP, a comeback story by secondaryacct on Wednesday November 17, 2021 @10:37PM (#29261721)
if needed=each time presence change.

So you d have an enormous graph with one node in the center refreshing presence for everyone. And end up worse.

Re:XMPP, a comeback story by IanSanders on Thursday November 18, 2021 @05:28AM (#29263987)
What do you suggest replacing it with?