Element (the primary Matrix software) definitely has Slack and Discord in its sights.
I don't think there are any serious "self-hosted Slack-like" contenders that are XMPP-based right now. You can piece components together (yay, standards!) and I did exactly this for the IETF's XMPP deployment recently. But it's far from being a cohesive easy-to-deploy product. Simply because nobody is building that right now. It takes time and resources and there's no money in it.[1]
People who do set out to build Slack clones (projects like Mattermost and Rocket Chat) and earn money don't have features such as federation on their priority list and don't build on top of Matrix/XMPP. They roll their own custom protocols and as far as I can see they are fairly content with that decision.
[1] There's even less money it, but nevertheless I am currently working on such a self-hostable "package" for XMPP. However rather than focusing on the team chat use-case (Slack/etc.) I'm focusing on personal messaging (WhatsApp/etc.): https://snikket.org/ if you're interested. It's possible I will broaden the scope one day.
EDIT: typo
The essential problem IMO is how to replace SMTP. No one has proposed and implemented an alternative, to my knowledge. So I decided to[1]. The current draft omits federation (although I wouldn't rule it out in all cases yet).
[1] https://github.com/networkimprov/mnm/blob/master/Protocol.md
> problem IMO is how to replace SMTP.
Sadly SMTP is probably one of the parts of Mail which have aged best. Enforcing the usage of some (currently by design optional) features wrt. authentication and similar at the cost of backwards compatibility and you have all you need from the delivery protocol.
BUT:
- IMAP and similar is much worse.
- Mail bodies are a big mess it's always fascinating for me that mail interoperability works at all in practice (again you can clean it up a lot, theoretically, but backwards compatibility would be gone).
- DMARC, DKIM and SPIF which handle mail authenticity have a lot of rough corners and again for backward compatibility are optional. Again it's not to hard to improve on but would brake backwards compatibility.
The main reason mail still matters is because it's backwards compatibility, not just with older software but also with new software still using old patterns because of the (relative to the gain) insane amount of work you need to put into all kinds of mail related components. But then exactly that backwards compatibility is what.
(Yes, I have read the "Why TMTP?" link and I have written software for many parts around mail including SMTP, and mail encoding. The idea that SMTP is at the root of the problem seems to me very strange. Especially given that like I mentioned literally every other part of mail is worse then SMTP by multiple degrees...)
EDIT: Just to prevent misunderstandings one core feature of mail is the separation of mail delivery and mail authenticity, in the sense that you don't need the mailman to prove the authenticity of a mail. At most the legal/correct/authentic delivery.
The opposite is also true.
TMTP also covers most IMAP/POP use cases. And it allows short, plain-text messages (see Ping) to make first contact with others -- necessary when that server has less restrictive membership requirements.
Authenticity is a double-edged sword. For certain confidential content, you want the recipient to know that it originated with the sender, but you don't want anyone else to know that in the event the content is leaked or stolen.
I believe the extinction of email for person-to-person & app-to-person correspondence is a foregone conclusion, due principally to phishing. The question is what should we do now, and the answer is clearly not chatrooms (which are of course useful in certain circumstances).
One email like protocol that properly handles this is NNTP.
Regarding the actual modes of discussion I was thinking of though, usenet and email are mostly the same.
For the most part, they are and many readers support both protocols (or at least they did in the past). The nice thing about NNTP is that it doesn't require maintaining a separate archive or having someone send you an mbox file to import. Just subscribing to the appropriate groups was sufficient (depending on the article retention policy).
Protocol: https://github.com/networkimprov/mnm/blob/master/Protocol.md
Why TMTP? https://mnmnotmail.org/rationale.html
Follow: https://twitter.com/mnmnotmail
And you can keep using all the normal MUA's on desktop and mobile.
Setting up a new server isn't easy unless you hire an outside service provider, and if you're willing to do that, Slack et al offer a nicer UX than the well known email/webmail clients.
Orgs with sufficient IT resources commonly do run internal SMTP servers.
Sorry my dude, but business runs on email. Saying lets get rid of it is as naïve as saying lets get rid of Excel. It's just not going to happen.
1) Set up a second TMTP service where customers and/or suppliers can create accounts, along with employees who need to interact with them.
2) Have some employees join a third-party service which is open to all involved in your field. There is a risk of phishing in this case, but:
. a) anyone you haven't previously contacted is limited to short, plain-text communications to you (see Ping in protocol), and
. b) such third-party services would typically charge a fee to members and impose a small cost per-ping, and
. c) you know that you're dealing with unknown entities with possibly malicious intent.
TMTP clients support active logins to accounts on any number of TMTP servers, just as browsers support multiple active connections to websites.
The same could be accomplished with email if you only allow connections to the SMTP and IMAP server from within the corporate network. That is, nothing external can connect to those servers, which is fine if it's only used for internal communication.
Ahem, I'm trying to bury email, not save it -- not unlike Slack :-)
Matrix is pretty straightforward on the server side of things, but the client UX is invariably mediocre. Vector—the official client—exemplifies everything that is wrong with Electron apps. Slow, clunky, poor UI, poor platform integration. With the default home server, it can take seconds for a message to go through. At least it’s far more customizable than Slack; it has an option for everything, which, as a power user, I quite like.
I haven’t tried Mattermost, but it looks like some of the important features aren’t FOSS, at which point it’s just another Slack as far as I’m concerned. I’ll gladly pay for support, but for SSO? Meh, might as well stick with Slack; at least everyone and their dog knows how to use it. (This is, of course, an opinion that stems partially from ignorance; I haven’t actually tried Mattermost, and if I do, I might fall in love with it. But my time is limited, and I can only evaluate so many products in a day.)
Not that Slack is much better here: their threading system has so many UI/UX issues. Ever had a thread with hundreds of messages? For your own sanity, I hope you haven’t. Ever tried to send an image to a thread from iOS? It’s possible, but only by pasting the image into the text field; the normal attachment button isn’t available, and Share buttons in other apps can’t send to threads. And, of course, the recent uptime issues.
That said, Element could certainly use less RAM, irrespective of Electron - and http://hydrogen.element.io is our project to experiment with minimum-footprint Matrix clients (it uses ~100x less RAM than Element).
Wow – congrats!!
What have been the most important architectural decisions to achieve this?
Edit 1: I want to love it; the design is everything I could ever hope for in a chat platform. I even tried to contribute to Vector, but it was such a mess that I eventually gave up.
Edit 2:
> That said, Element could certainly use less RAM, irrespective of Electron - and http://hydrogen.element.io is our project to experiment with minimum-footprint Matrix clients (it uses ~100x less RAM than Element).
I'm not sure why this is a priority. Techies complain about RAM usage a lot, but if we have to choose between performance+power and a small memory footprint, we're going to choose the former almost every time. Take Telegram, for example: they have a bunch of native clients that perform amazingly well, although they do gobble RAM. Most of my technical friends use it as their primary social platform. It's not without issues, but it's really hard to go from something like Telegram Desktop or the Swift-based macOS Telegram client to Vector. And those clients aren't made by large teams--most (all?) first-party Telegram clients are each maintained by a single developer, if I'm not mistaken.
Does Element refer to the ecosystem as a whole, including EMS? The primary client? The core federation? It’s not obvious from a casual visit to element.io. I suppose if I said “Element web app,” that would be fairly clear, but I’m still in the habit of saying “Vector” from the days of Riot.
The protocol is still called Matrix.
Anyway, my point stands: Element Web/Desktop feels fairly unresponsive compared to something like Telegram Desktop. It looks so much nicer now, the UI layout is great, and it’s far more powerful than Telegram—yet, I can’t help but feel like I’m swimming through molasses even when dealing with moderately-sized groups. Try clicking around on different groups rapidly; you’ll likely find that you have to wait several seconds for the UI to update.
XMPP is, well, extensible. If things don't match in the clients then that particular feature just doesn't work. These days all the clients pretty much try to match the feature set of Conversations. That applies to the servers as well. There is a server tester for that:
* https://compliance.conversations.im/
So the tooling is pretty good these days.
The great thing about XMPP is that basic messaging always works. That stuff is just too simple not to.
It even sent a 20 Mo MP4 like a champ, while Conversations sometimes chokes on not that high resolution photographs...
I run a small homeserver and use it to communicate with a group of about 20 friends. Most of them aren't "technical" people. We use it mostly for chatting and image/video sharing. We never use live calling (audio or video).
There have been a few bugs in the mobile apps, but for the most part, everything has been working fine.
The biggest issue is the UX. It's not as polished as the big players.
Like I said, it's close, I just don't think it's there yet.
The only thing that's a little cumbersome is requiring them to enter a custom server URL when the register/log in for the first time.
> The free Discord plan provides virtually all the core functionality of the platform with very few limitations. Free users get unlimited message history, screen sharing, unlimited server storage, up to eight users in a video call, and as many as 5,000 concurrent (i.e., online at the same time) users.
For a lot of small communities that aren't focused around commerce of any kind, Discord's free offering blows Element Matrix Services out the water. It's a non starter. If I could create a server with feature parity to Discord's free server, any new community I'd create I would definitely jump on EMS in a heartbeat, and I'd start trying to recreate communities currently within Discord, to be on EMS.
So like a very normal progression for Discord servers is that some niche sub-community wants to gather, and so they create a free server, and people join and there's all kinds of rich content that gets posted and curated and great discussions and then as it gets bigger, people running the community or people who want to support the community will boost the server with Discord Nitro for additional features like more slots for custom emojis (I can't communicate enough how important of a feature this is to Discord's success, even though it seems like minor window dressing).
That kind of model is what would justify a server starting to shell out money every month for EMS. I would note that Discord's pricing for this kind of level of community is tiered and not a per-user thing. You unlock more features based on how many users are paying for Nitro, going up a tier based on breakpoints of 2/15/30 Nitro Boosts per month. It doesn't cost more to have a tier 3 server if you gain more users. This is a big deal for fostering growth and unseating incumbent social networks (which is what Discord and Slack are).
Just some thoughts. I really want stuff like Element/Matrix to succeed!
and use it to communicate with a group of about 20
friends. Most of them aren't "technical" people.
I'm insanely curious about the human side of things here. How did you get them to buy into this idea in the first place? That sounds like quite an achievement.The non-technical folks in my life generally struggle with paths of least resistance (iMessage, etc) and it's hard to imagine getting them onto some alternative platform/protocol.
Previously, we were mostly using group texts or Snapchat/Instagram to communicate, so the biggest selling point was the fact that we can share full quality pictures and videos between iOS and Android people.
That doesn't happen for no reason. The vast majority of open source products I've used have terrible usability. I simply don't want to use them. I don't want to be beholden to corporations and walled gardens, but for me, the existing alternatives are worse in too many ways.
Not when it's down.
No hard data though. My mail server only ever went down when I upgraded the server and didn't check that everything was still working right away, or similar maintenance induced incidents. It never went down by itself.
Such systems only ever go down unpredictably on HW issues, or when overloaded/out of resources. Neither is very likely, because you're not trying to grow your service in any sense similar to VC backed enterprises. Most of the time it has constant very low load and resource use. And you can simply stop introducing changes to the system if you need more stability for some time. (stop updating, for example)
There might have been political reasons why google dropped XMPP, but it would also make sense as a purely technical decision.
This is plain untrue. Yes it was invented a long time ago, but thanks to the extensibility it has evolved over time just as the way people use it has changed. This evolution is a healthy and necessary part of an open ecosystem.
I know it frustrates people that modern features don't work in stagnated clients such as Pidgin and Adium, but modern clients support all the things you would expect.
Servers and mobile clients have supported mobile-friendly traffic and connection optimisations for many many years now.
> There might have been political reasons why google dropped XMPP, but it would also make sense as a purely technical decision.
Google contributed extensions to XMPP, the same way they contribute to other internet standards. I think they were quite comfortable with this. The XMPP-based Google Talk was their longest-running messaging solution after all...
What makes you think so? If Conversations was draining my battery, I would have noticed by now, I'm pretty sure that Facebook Messenger is worse in this aspect...
Has this been fixed in XMPP?
That's true for email also. So that's not an evidence for anything lacking in xmpp. The plain fact is that Google killed it due to their greed.
Scrolling back is atrociously slow, and it doesn't even seem to have a search feature !
I'd take XMPP alternatives like Conversations, Jitsi, Pidgin any day ! (And Element of course.)
The only thing "killed" XMPP was that proprietary made money, XMPP didn't.
Apart from that, it's alive and well. See Conversations for android, Prosody for server.
Mattermost is certainly not what I meant. That's just trading one Slack for another.
From my experience, when I self-host stuff it's a lot faster (more server resources) and never had any downtime (server doesn't simply go down for no reason).
We ended making the switch and committed to Discord. We're now looking at Rocket.chat as a backup in case Discord goes down. But Slack is now completely out of the picture for our team.
I’ve advocated for an idea where Mattermost is to be used as a “bunker” where it is hosted on a raspberry Pi (or somewhere else) and acts as a digital bunker if your critical infrastructure (slack, teams, exchange?) is compromised somehow.
I know it's a tough spot, but if it were usable from GitLab with zero config that would be great for fallback.
To enable Mattermost, you can add the Mattermost external URL in the config file, and run `sudo gitlab-ctl reconfigure`. I'm wondering if that's something you've tried? https://docs.gitlab.com/omnibus/gitlab-mattermost/#getting-s...
* You get notifications for channels for which you have suppressed those notifications
* Some channels are marked as having new notifications, when they haven't
* Notifications for new messages in threads you are involved in are quite hard to find (horrible UX)
* Some UX choices are very confusing (you get a column of options related to notifications, and for some, the left option is the one leading to more notifications, for some the right option)
* There are some overlapping features that lead to inconsistent usage (channels vs. discussions vs. threads)
* Threads are hard to read, because follow-ups in threads are shown in a smaller font size. You cannot increase the font size at all in the desktop application
.... and so on.
Also, I tried to submit some bugs, but for that I'd need to have some information which only the admins have that run this instance, and in the end it was too much effort to get all that information together, so I didn't even bother.
I'm in a slack workspace that is constantly notifying me of a thread, but I can't make it read. Maybe it has something to do with the free messages limit. So the message is there, but cannot be accessed. Annoying as hell. I thought about submitting the bug to Slack, but then just let it go, and probably we'll just move to Signal or something.
But I signed up and will keep it on my radar as it matures.
How often is Slack/Discord down? I mean it's not perfect, but I really honestly don't think I could match their uptime by self-hosting, as well as more on-call rotations for something that's not core product.
I very much prefer that for something that isn't core product, if it goes down I need to do exactly nothing for it to come back up, and that the engineers at Slack will be starting to work on it likely before I even realize it's down.
Hardware is quite reliable these days. And updates can be scheduled to be at a convenient time for the team.
If there are other people on the team that have _some_ technical skills then they can fix it..
IRC lacks quite a few features compared to other solutions, but the reduced complexity does bring very low operational complexity.
It's just not a business-friendly tool.
I'm an engineer and personally I'm fine with IRC, I'm just trying to be realistic here.
Also there are plenty of modern web clients for IRC, such as https://thelounge.chat/ or https://kiwiirc.com/ (which is supposed to work on mobile too).
Reality check: Most people don't use email programs anymore.
Also how do you get IRC to sync all conversation data, history, between your several desktops and phones, how do you send files, make calls, and thread conversations?
And you are starting to move goalposts here.. first it was uptime, then it was operations and now it's features...
And what about those web based IRC solutions? They are even easier to use than slack, have combined history, file sharing, etc.
Add to that the SaaS propaganda that hosting literally anything yourself is just too hard (it really isn't). Or this notion people are just too stupid to deal with anything more than the simplest possible web interface - Really? what do those people even do? Stare at Notepad all day? Of course not. They stare at various complicated software packages ranging from CAD, $spreadsheet abominations, SAP to various Adobe software packages. Sprinkle in a bit of hype for the latest new thing and presto.. </rant>
Guess you are not in enterprise.
If you are running networks and software on site, and they are business-critical, you have people and a plan for this. Or you don't, and suffer the consequences.
When self hosting you can get away with simpler systems that ends up being more stable and have higher up times for lower effort.
The reason you see cloud providers having issues is not because the thing is difficult, but because doing anything at huge scale ends up being difficult.
This is such a common misconception. The services I self-host was configured by me, if anything goes down (which they very rarely do), I know the exact cause and have it fixed in minutes. When some company's cloud service goes down I'm completely at their mercy. I also spend very little time on maintaining these services, just security updates, which are mostly automated.
Bottom line, maintaining and self-hosting services that has 1 or a few users is much less complex than services with millions of users. Hence, my uptime is better than Google's, Amazon's, and Azure's, etc.
1. There is virtually zero user-facing documentation. Need to know how to backup keys, verify another user, or what E2EE means? Ask your server operator. Basically the onus is on operators to document this stuff for their users. Except the stuff we're documenting is hard even for server operators, and especially challenging to document in a way that both nontechnical and technical users can understand.
2. Because this stuff is challenging even for more technically minded users to understand, it leads to a kind of burnout for interested non-technical users where they learn all they can about some feature and how it works at a high level from out of date random blogs, try to use the (complex, multi-step) feature, but then something won't work, and it isn't be clear whether it was because the user did something wrong or because the clients or server implementations are broken
3. Issues where core functionality is broken (e.g. two mutually verified users on my homeserver haven't been able to talk to each other in months -- see [1], [2], [3]) languish for months with zero response from maintainers.
4. While core functionality is both broken and undocumented, the maintainers announce rabbit hole features that no one asked for and seem very much like distractions, like their recently-announced microblogging view/client[4]
In short the Element maintainers have shown little interest in making the platform accessible to the people who need its differentiating features the most, and have prioritized the "mad science"/technical aspect of their platform at the expense of the human element (end-users and operators).
It'd be cool if Element used their resources to hire some UX folks and community advocates whose sole focus is addressing the horrid accessibility of their platform. I think most users would rather see that than further "mad science".
[1] https://github.com/vector-im/element-ios/issues/3762
[2] https://github.com/vector-im/element-ios/issues/3572
Also, it seems the FAQ answers several of your points: https://element.io/help
That FAQ is a great start, but it's not sufficient for non-technical users. It's not easily searchable, it doesn't provide screenshots, and it doesn't go into enough detail for each item (e.g. describing what can go wrong + troubleshooting).
Thanks for pointing it out though
it also doesn't take gigs of memory on client devices.
I'd like to, but unfortunately I don't feel like I can in good faith. Matrix is woefully immature, and suffers from a lot of issues, but I think is closer to being a functional Slack/Discord alternative. XMPP is much more mature, and works very well for chat, but doesn't have a nice package that does all the Slack stuff--at least not that I'm aware of. I'd love to be proven wrong there. I know it can be done, but if it can't be deployed quickly by an already overstressed team member, what chance does it have?