News for hackers. Stuff that matters.
Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @05:07PM (#11300687)
A hundred different chat systems mentioned here. None of them compatible with one another. Well, I guess a couple of them have IRC gateways, but you have to actually set those up.

Gee, wouldn't it be nice if you could pick and choose your UI? Pick and choose your "integrations", your "plugins", your client, etc without having to lose your entire userbase, history, contacts?

Some sort of open protocol.

Urgh, I've been working on this lately and the entire field is depressing. Between one side that doesn't understand the legitimate need for open protocols, and another side that doesn't understand why IRC isn't the end-all-be-all of group chat (and why UX matters), it's just people talking past each other.

Every new attempt at making "the slack killer" makes this problem worse, because it comes with its own protocol. Its own users. Etc.

PS: If somebody has free time to work on an open source multi-protocol group chat gateway, email me, I have something started.

Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @05:25PM (#11300805)
Here's a timeline we put together of about 60 incompatible chat services: https://cdn.sameroom.io/chat-timeline.pdf

Our service Sameroom (https://sameroom.io) is a commercial solution to the chat incompatibility disaster, not too different from an AC adapter thingy with a bunch of different plugs, one or two per continent.

After implementing the protocols of quite a number of these systems (~20), we're seeing a scary acceleration toward further incompatibility. It's good for our business, but really pretty bad for the world.

Re:Open Sourcers Race to Build Better Versions of Slack by takeda on Wednesday March 16, 2016 @10:52PM (#11302295)
This is so disappointing.

Especially Jabber/XMPP was created to fix that issue by standardizing the protocol. The idea is that IM would be like email - no central server.

Others (GTalk, Facebook, WhatsApp, HipChat to name few) started implementing it but did not enable server to server communication (ok GTalk did, but as soon as they became popular they intentionally broke it with Hangouts).

So now we have bunch of services that use same protocol but still can't communicate with each other, the XMPP seems like it is dying.

A while ago there were plenty of server-side transports to help migrate to XMPP, but it doesn't seem like they are no longer being developed.

I don't understand why no one cares anymore about decentralizing and standardizing IM.

Re:Open Sourcers Race to Build Better Versions of Slack by zamalek on Thursday March 17, 2016 @05:10AM (#11303296)
The problem is that the IM protocol space seems very opinionated. "XMPP? XML sucks!" "Web sockets? Web sockets suck!" "JSON? JSON sucks!" "Binary? That prevents telnet, binary sucks!"

People do care, it just seems as though they care about their pet peeves more.

Re:Open Sourcers Race to Build Better Versions of Slack by iheartmemcache on Thursday March 17, 2016 @09:49AM (#11304285)
That "XML" sucks thing is why we don't have single-sign on (YuBiKey is doing a great job getting half way there though). We keep on reinventing the wheel re: authorization & authentication despite the fact that there are well-defined, cryptographicly secure[0]. SAML2[1] has been standardized, implemented, operates in a similar fashion to SMTP/Kerberos (i.e., your identity is established (i.e. authentication) purely through a trusted-source), after which you can proceed.

  * This eliminates the whole issue for users (I don't want to have to remember 40 million passwords, and I don't want to rely on Google as my SSO). 
  * It helps the developer because all he has to do is implement one protocol (rather than make 8 different accounts to register passport.js with).
  * It helps the tin-foil hatters like me, because I can revoke access from any source at any time & if my account gets compromised I call my buddy Bill hosting the IDP and invalidate my credentials.
It's SMTP mail back in the late 90s when your buddy ran his own server as your primary MX, and his buddy as a secondary MX (i.e., you host an IdP similarly to hosting a mail server).

I've been working on this problem on and off for about 10 months, the last 4 of which have been fairly rigorous. I've got a solution for the "buddy" willing to host the IdP. https://github.com/bergie/passport-saml is a low friction add-on to your node app for the developers. The last piece of the puzzle is dev adoption (i.e., getting traction on par with LetsEncrypt) to actually spend 5 minutes for that passport-saml stuff.

The other issue is developing an easy solution (easier than SSO with Google) done by mobile App authentication. User-flow is basically: you click "Login with <name>", the IdP forwards the message to your mobile via APN, and you swipe left or right to auth. That's the last technical barrier (user adoption is 80% of the problem). I need to take ~3 mo off and pony up high-five, low-sixes in fees re: getting a great UX guy (I'm abysmal [[if you're great at UX, contact me]]), lawyers, liability insurance, a security firm to audit, etc (there's already a monetization model for this venture, though the whole thing will be GNU AGPL[3] / free for universities / research institutions, etc.)

[0] Like, in the formal mathematically proven sense, both in concept, and implementation (though which one slips my mind).

[1] http://wso2.com/library/articles/2010/07/saml2-web-browser-b...

[2] Encryption is basically a subset of privacy, which I think we have a constitutional right to, though other people object.

[3] The whole point is to do this first as a human good, but profiteer off expensive licenses a la Exchange modules, et al. I don't really want to shop around Sandy Hill but if any of you successful Angels (who are historically privacy advocates, I'd give Karsten Nohl or DJB (or hell, PG if you're reading this message haha, 30% and 2 seats in a second, but Larry Ellison could offer 3MM for .1% and that meeting would conclude rapidly) are interested, e-mail me me -- we'd be able to go to market a lot quicker with 2 or 3 FTE's. I'm going to complete this anyways, but we're fighting cryptowars again so there might be a rush.

Re:Open Sourcers Race to Build Better Versions of Slack by zamalek on Thursday March 17, 2016 @09:53AM (#11304318)
We integrate pretty deeply with SAML and, yes, it's awesome tech. It makes complete sense that XML was used because real namespaces are important for claim extensibility. Sadly, "XML sucks!"
Re:Open Sourcers Race to Build Better Versions of Slack by zanny on Thursday March 17, 2016 @12:12AM (#11302526)
People do care. I'm working on a Qt Matrix client for KDE / maybe eventually Telepathy.

The problem is non-techies do not understand why email is good while messaging is bad. There is no indication as to where these problems would come from, and they have no insight into why the problem happens in the first place, while they gleefully eat up iMessage / Facebook Messenger / Skype / Hangouts / Whatsapp.

Re:Open Sourcers Race to Build Better Versions of Slack by teekert on Thursday March 17, 2016 @04:39AM (#11303198)
Because to a non-techy, running a messaging server is just as out of reach as running a mail server. It's just too difficult.
Re:Open Sourcers Race to Build Better Versions of Slack by iheartmemcache on Thursday March 17, 2016 @09:52AM (#11304310)
Lots easier than before. Off the top of my head: 1: You have more than enough 'power users' who keep their machines on 24/7 for torrent seeding. VirtualBox is free & those guys are fairly security conscious as a result of the inherent defensive nature of torrenting (MPAA litigation, etc). Give them a VBox server they can run and they'd happily do it. Now grandma can use his server no problem, in the same way that you only needed 1 techie in your group (pre-AOL) to host an SMTP server.

The real difficulty is convincing grandma that she needs to leave "AOL".

Re:Open Sourcers Race to Build Better Versions of Slack by teekert on Thursday March 17, 2016 @11:16AM (#11304848)
That is oversimplifying, for example what happens when Microsoft seems to make your email to your brother in-law disappear and they tell you they have no control over their spam filters? Or when you sent out birthday invites and every Yahoo user you know presents you with an automated message that your server has been banned forever from sending them mail? So you set out to get SPF records, get DKIM working, get certs... and it doesn't help.

Also, running torrents is different from getting all the DNS set up and forwarding all the ports. And even then, where I live, out of the 10 or so fiber providers I can choose, I know that only 2 allow me to do anything with port 25.

It can be a Pita.

Re:Open Sourcers Race to Build Better Versions of Slack by takeda on Thursday March 17, 2016 @05:08PM (#11307625)
One doesn't need to run own server, they can use one of the public ones:

https://xmpp.net/directory.php

Similarly how it's done with mail.

Re:Open Sourcers Race to Build Better Versions of Slack by pjc50 on Thursday March 17, 2016 @05:32AM (#11303365)
GTalk did, but as soon as they became popular they intentionally broke it

This is the thing. If nobody owns it, nobody can exploit it; if nobody can exploit it, it can't get VC funding.

Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @10:56PM (#11302308)
I think because it's a "nice-to-have", not a "must-have".

(We have SMTP, a must-have.)

There is no such thing as a "nice-to-have" universal protocol.

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @05:32PM (#11300845)
Thank you, that PDF is incredibly depressing.

By the way, well done on sameroom. I came across your product while working on the stuff I mentioned and it looks like an incredibly useful thing. I really wish you would open source it, though. (Or at least open source some libraries for talking to the protocols you deal with. Especially Hangouts.)

Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @05:39PM (#11300883)
Thank you!

We're so paranoid about getting shut down by Google (Hangouts, no API), Microsoft (Skype, no API), and Facebook (Messenger, no API) that we're not only not open sourcing any of our implementations, we're not even providing a commercial API in fear of having to deal with some form of abuse.

That said, the night is young. We hope to eventually gain enough leverage to get the big guys to listen to us.

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @05:46PM (#11300938)
Ok. It's kind of good to hear the barrier to open sourcing is more legal than commercial. That means it's fixable :)

Can I ask you some details on how you went about working with Skype and Hangouts? Did you do the reverse engineering yourself? These are protocols several popular open source projects have been trying to support for years and haven't succeeded.

Can I also ask you about your tech stack/languages you used for Sameroom?

Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @06:01PM (#11301019)
We implemented them ourselves, and it took quite a bit of work.

Our magic weapon is Erlang, which is a wonderful language for working with protocols, and a wonderful runtime for working with many things happening at the same time.

The other magic weapons are the Cowboy webserver and the Gun http client -- both written by Loïc Hoguin. Loïc doesn't work for us, but we've been sponsoring him financially for over two years (https://medium.com/@abs/what-we-learned-from-sponsoring-an-o...).

Our frontend doesn't really matter as much, since it's just a dashboard, but we use TypeScript with Flux and React (it's been great).

Re:Open Sourcers Race to Build Better Versions of Slack by derefr on Wednesday March 16, 2016 @11:36PM (#11302419)
I was just considering advocating within my employer (Cloudant, i.e. clustered-CouchDB-as-a-Service) to back Loïc too. CouchDB is still stuck in the past with Mochiweb, and while switching that out for Cowboy 1.X would be a nice-to-have, getting our hands on a stable release of Cowboy2 (i.e. "the HTTP/2 version") would let us optimize Couch's replication story immensely. I've just been trying to set up a single trivial Event Stream HTTP/2 loop_handler with the Cowboy2 master, and it's really just not there yet, so we could probably give him a push.
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @11:53PM (#11302473)
That would be great - I can put you in touch if you're not already ([email protected]).
Re:Open Sourcers Race to Build Better Versions of Slack by seivan on Wednesday March 16, 2016 @09:25PM (#11302005)
Care to share your webpack config for getting React and TypeScript to play ball? I'm collecting a bunch to try to figure out what works best.
Re:Open Sourcers Race to Build Better Versions of Slack by jlarky2012 on Wednesday March 16, 2016 @09:51PM (#11302094)
That's actually interesting topic :) Right now we use gulp and gulp-typescript. So far, I like approach that is used in https://github.com/Microsoft/TypeScriptSamples/tree/master/e... but I'm not sure if we are going to use it any time soon. As far as gulp config, what we use is pretty much close to gulp-typescript README examples
Re:Open Sourcers Race to Build Better Versions of Slack by seivan on Thursday March 17, 2016 @03:24PM (#11306861)
Thanks! Yeah it's hard to find good UP TO DATE reading material on this. I genuinely appreciate this.
Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @09:41PM (#11302068)
You can take a look at ours: https://github.com/hearthsim/joust
Re:Open Sourcers Race to Build Better Versions of Slack by zanny on Thursday March 17, 2016 @12:10AM (#11302520)
There are actually FOSS implementations of Hangouts wrappers (Hangups) Skype wrappers (SkypeWeb for libpurple) and Facebook wrappers (purple-facebook) and even a telepathy wrapper for the Facebook protocol.

Nobody is getting sued yet, and Pidgin supports all of them nowadays.

Re:Open Sourcers Race to Build Better Versions of Slack by vvanders on Wednesday March 16, 2016 @05:50PM (#11300956)
Re:Open Sourcers Race to Build Better Versions of Slack by pyre on Wednesday March 16, 2016 @06:56PM (#11301326)
Or how AOL's Oscar protocol got updated at one point to start requesting chunks of the AOL IM binary as authentication against open sources clients (because they couldn't distribute AOL's binary with their client).
Re:Open Sourcers Race to Build Better Versions of Slack by khc on Thursday March 17, 2016 @04:43AM (#11303210)
Do you have a source for this?
Re:Open Sourcers Race to Build Better Versions of Slack by quadrangle on Wednesday March 16, 2016 @09:27PM (#11302021)
Sounds like you just need the AGPL
Re:Open Sourcers Race to Build Better Versions of Slack by WoodenChair on Wednesday March 16, 2016 @09:45PM (#11302077)
Yeah because if you open source something under the right license, no abuse happens... What percentage of GPL violators has the FSF prosecuted? Or anyone else for that matter?
Re:Open Sourcers Race to Build Better Versions of Slack by chei0aiV on Thursday March 17, 2016 @04:21AM (#11303126)
The FSF isn't about prosecuting people, but about getting GPL compliance.

https://sfconservancy.org/copyleft-compliance/principles.htm...

Unfortunately working on GPL compliance is expensive, I would encourage anyone who cares about copyleft to support the Software Freedom Conservancy's efforts.

https://sfconservancy.org/supporter/

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @09:43PM (#11302070)
Licensing has nothing to do with this.
Re:Open Sourcers Race to Build Better Versions of Slack by yborg on Wednesday March 16, 2016 @06:15PM (#11301093)
Not sure what is so depressing here. According to the chart, a total of nine listed networks died in 50+ years, this is actually astoundingly low.

And all IRC implementations are "interoperable" in the sense that a single client that implements the protocol can connect to any of them, this chart seems to be padding things a lot by calling every IRC server network a "chat service". A number of them also have or have had interconnects with each other in the past.

Re:Open Sourcers Race to Build Better Versions of Slack by tlrobinson on Wednesday March 16, 2016 @11:52PM (#11302470)
Surely there were networks that went defunct before 2011. That's probably when they started paying attention.
Re:Open Sourcers Race to Build Better Versions of Slack by chejazi on Wednesday March 16, 2016 @08:27PM (#11301781)
The depressing aspect is the accelerating fragmentation caused by more custom protocols entering the game. Supporting them all as the space grows will only get more cumbersome.
Re:Open Sourcers Race to Build Better Versions of Slack by amai on Thursday March 17, 2016 @06:29AM (#11303544)
I think Signal from https://whispersystems.org/ deserves to be mentioned, too.
Re:Open Sourcers Race to Build Better Versions of Slack by andrey_utkin on Wednesday March 16, 2016 @05:43PM (#11300919)
Out of curiosity: are you leveraging such software as spectrum (spectrum.im) and libpurple? These pieces seem to be a good fit for internal gears of your product.

Good luck, you are working on an important problem.

Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @06:16PM (#11301106)
Hi Andrey - we're not using spectrum.im or libpurple (but are aware of both). We saw that libpurple now supports Telegram, so we may borrow some ideas from them :)

Thank you!

Re:Open Sourcers Race to Build Better Versions of Slack by chei0aiV on Thursday March 17, 2016 @04:33AM (#11303174)
Please look at Telepathy too.
Re:Open Sourcers Race to Build Better Versions of Slack by peteretep on Wednesday March 16, 2016 @07:40PM (#11301554)
It doesn't look like you provide quite what I'm after, which is a single OS X / iOS app that seamlessly connects to all of those, while supporting the differentiating features of each. Would easily shell out $30 for such a thing, and maybe even $10 a month for a licensed version. Setting up "tunnels" between client installs probably gets me fired.
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @08:20PM (#11301754)
We focus on connecting teams vs individual productivity. Check out http://meetfranz.com/ for a single app option.

About getting fired - not if we sell our solution to your IT :) They might like it, actually, since they can a) provide something useful to their user base and b) get a better handle on the chat situation.

Re:Open Sourcers Race to Build Better Versions of Slack by voltagex_ on Thursday March 17, 2016 @12:23AM (#11302561)
You find yourself in the same position that Trillian did, what, 15 years ago?

Thanks for giving it a shot, at least. It's getting to the point where if I wanted to keep in touch with most of my friends all the time, I'd need to carry two or more devices (thanks for keeping iMessage closed, Apple).

Re:Open Sourcers Race to Build Better Versions of Slack by mwfunk on Thursday March 17, 2016 @12:35AM (#11302604)
iMessage works fine with SMS and MMS.
Re:Open Sourcers Race to Build Better Versions of Slack by molmalo on Thursday March 17, 2016 @02:13AM (#11302835)
Which is absolutely irrelevant. It's expected from a mobile phone to be able to send and receive SMSs. But SMSs can take several minutes to reach its destination in some areas. Long messages are split. And the experience as a whole is totally different. And of course, most people pay per message (individually or as a pack), at least globally. So, if someone wishes to send a message, it shouldn't matter what OS or app is being used by the other party. IF GP uses an Android phone as its primary phone, it may be problem if he is trying to chat with someone who uses only iMessage.
Re:Open Sourcers Race to Build Better Versions of Slack by akerro on Thursday March 17, 2016 @04:22AM (#11303128)
Facebook, Google chat and some others XMPP based were compatible and you could chat with Facebook users without having facebook account, but they closed their services making it all incompatible.
Re:Open Sourcers Race to Build Better Versions of Slack by ldarby on Thursday March 17, 2016 @11:32AM (#11304958)
That PDF suggests Zulip died after being acquired by Dropbox, but that's not the case, they opensourced it: zulip.org
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Thursday March 17, 2016 @12:52PM (#11305641)
We decided to only include cloud services, so we weren't sure what to do with something you have to run yourself. That said, the line is blurry, we will fix.
Re:Open Sourcers Race to Build Better Versions of Slack by donatj on Wednesday March 16, 2016 @07:57PM (#11301633)
The timeline is missing the fact that ICQ converted to Oscar after the AOL acquisition and users on both networks can talk.
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @08:23PM (#11301767)
Thanks - will fix.
Re:Open Sourcers Race to Build Better Versions of Slack by alexvoda on Thursday March 17, 2016 @02:03PM (#11306237)
Would you mind releasing the source (SVG? Illustaror?) of that very depressing chart under an freedom respecting licence (CC-BY? CC-BY-SA?) and hosting it on something like Github/Bitbucket/Gitlab/Codeplex? I would like to contribute by updating it. Also, I would like to link to it on Wikipedia.
Re:Open Sourcers Race to Build Better Versions of Slack by Apocryphon on Wednesday March 16, 2016 @11:02PM (#11302324)
As it turns out, the tower of Babel is completely self-perpetuating.
Re:Open Sourcers Race to Build Better Versions of Slack by txdv on Thursday March 17, 2016 @04:12AM (#11303093)
How have you implemented skype?

Have you written a stand alone library which communicates with skype?

Re:Open Sourcers Race to Build Better Versions of Slack by creshal on Thursday March 17, 2016 @05:03AM (#11303275)
Skype has a web-based API nowadays that's used by quite a few third-party clients.
Re:Open Sourcers Race to Build Better Versions of Slack by brazeon on Wednesday March 16, 2016 @05:30PM (#11300834)
Do you plan on open sourcing plugs or any parts of your system?
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @05:41PM (#11300903)
You can see what happens to OS projects that work with WhatsApp, Skype, and some of the other heavily guarded walled gardens -- cease & desist orders, carried out by GitHub.

We have absolutely no interest in joining that list, which unfortunately means the answer - for the time being - is no :(

Re:Open Sourcers Race to Build Better Versions of Slack by cyphar on Wednesday March 16, 2016 @05:58PM (#11301002)
Right, but surely if you're selling it that's much more incentive for a cease and a desist (regardless of whether it is free software).
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @06:05PM (#11301040)
Actually I don't think so. The main thing the owners of big networks are afraid of is spam.

We provide a ridiculously expensive way to connect, say, Slack to Skype, which is ==great for Skype==, from almost every possible angle. It means Skype users can stay on Skype instead of switching to someone else's Slack team.

Re:Open Sourcers Race to Build Better Versions of Slack by taurath on Wednesday March 16, 2016 @10:03PM (#11302124)
In that case great for skype, bad for slack. Someone, somewhere will decide they'd rather force those customers to use their service/client, or else they'd have an open protocol in the first place.
Re:Open Sourcers Race to Build Better Versions of Slack by rekoros on Wednesday March 16, 2016 @10:16PM (#11302172)
There's a huge compliance problem with working in someone else's Slack team -- you own none of the resulting data and you don't control access on your end.

Email is much better in this way, since everyone gets a copy.

With Sameroom you essentially retain the carbon copy semantics of email, but with real-time collaboration. So, I'd argue that it's great for Slack.

Re:Open Sourcers Race to Build Better Versions of Slack by liveoneggs on Thursday March 17, 2016 @01:25AM (#11302738)
you left out ICB
Re:Open Sourcers Race to Build Better Versions of Slack by ex3ndr on Wednesday March 16, 2016 @05:10PM (#11300704)
We are actually trying to make open protocol in https://actor.im with federation with https://matrix.org. It seems that only Actor and Matrix understand importance of federation.
Re:Open Sourcers Race to Build Better Versions of Slack by e12e on Wednesday March 16, 2016 @08:50PM (#11301879)
Thank you for mentioning actor - I must have missed it the last time various open (federated) solutions were discussed.

It looks like an interesting platform, but I was a bit surprised by what I found at:

https://developer.actor.im/docs/encryption

"MTProto v2 Rev3 enables encryption support to replace or enchanse TLS one." So far so good - here's someone that's built on top of NaCl and built a legacy-free encryption system on top of well tested, known primitives, I thought.

But: "Actor encrypts message with US encryption and then again encrypt with Russian encryption that in result guarantee absolute encryption streight" (sic).

Uh. This sounds like applying a "political" reasoning to layering security: AES might be backdoored by NSA, GOST(?) might be backdoored by the FSB -- but using both, only double-agents will be able to foil our encryption!

While the truth is probably that you're know vulnerable to buffer overflows or other more mundane software errors in the now double-sized encryption code -- and get none of the (potential) benefits of a clean, modern architecture on top of just NaCl?

In addition, it appears you also use curve25591, possibly an implementation derived from NaCl:

https://developer.actor.im/docs/securing-server

"For secure communications Actor Server have to be configured with Curve25519 keys. To generate them, use actor-cli util:"

So that's three codebases worth of bugs, rather than one. Any plan to simplify this part?

Re:Open Sourcers Race to Build Better Versions of Slack by ex3ndr on Thursday March 17, 2016 @06:12AM (#11303495)
Our encryption is based on Java and everywhere we check buffer overflows and possible timing-attacks. Hopefully we use AES and SHA256 that is easy to validate. Also Instead of using N code bases (one for each platform) we reduce it's amount by conversing sources from java to other languages for every platform.

US-based encryption is taken directly from BouncyCastle, time-proven library for Java. Russian implementation can contain bugs, yes, but i can guarantee that there are no timing or overflow attacks possible. Even when we will have broken russian layer, us layer will be safe. Also we are on the way to make everything verified by 3rd parties and seal cryptography parts.

For key exchange we use curve25519. Implementation is taken directly from Signal and this app is trusted by majority.

We

Re:Open Sourcers Race to Build Better Versions of Slack by e12e on Thursday March 17, 2016 @06:52AM (#11303601)
Ok. But why use AES and GOST? AFAIK they're both based on feistel networks? If one were to layer two symmetric encryption schemes today I'd think one would go with something like salsa[ed:chacha] and keccak (in authenticated encryption mode) -- and it would probably still be a bad idea, adding much more complexity than security?

[ed: Oh, and here I go, proving that non-cryptographer should be careful about what they say wrt encryption, apparently AES isn't a feistel cipher? http://crypto.stackexchange.com/questions/10605/why-is-aes-n... ]

> Also Instead of using N code bases (one for each platform) we reduce it's amount by conversing sources from java to other languages for every platform.

It's a pretty strong claim to say that you take three different encryption primitives, implemented by other people in java, and translate them to other languages -- and can also be sure there are no timing attacks? (Or oracles, or resulting errors)?

Surely it would be easier to stick to as few lines and layers as possible if you want it to be easy(er) to audit?

(So simple there's obviously no mistakes, vs so complex there's no obvious mistakes and all that).

Re:Open Sourcers Race to Build Better Versions of Slack by ex3ndr on Thursday March 17, 2016 @03:23PM (#11306851)
I will let you know when we will complete audit of our crypto.
Re:Open Sourcers Race to Build Better Versions of Slack by espadrine on Thursday March 17, 2016 @10:19AM (#11304467)
You're not helping your case. You make it sound like you are using bricks without knowing what their strength is.

It's a bit like saying you made a boat out of wood for buoyancy and filled it with sand in case the wood catches fire. Sure, a plank of wood doesn't sink, but too much sand can make it sink, and sand won't be of much help if there is a fire. A wooden boat is better than one filled with sand.

I'm not sure how the fact that the boat was made in Java is relevant, either. If anything, it makes it harder to prove that there is no timing attack, as the JIT could do unexpected things, long after the program has started.

Re:Open Sourcers Race to Build Better Versions of Slack by e12e on Friday March 18, 2016 @08:02AM (#11311179)
Agreed. One OT nitpick: The fire-quenching properties of sand is actually somewhat different to that of water - eg in case of burning fuel/oil.
Re:Open Sourcers Race to Build Better Versions of Slack by lewisl9029 on Wednesday March 16, 2016 @05:23PM (#11300789)
I haven't looked into actor much yet, but the last time I looked at Matrix, my main concern was how much implicit trust users needed to place in homeserver providers (messages and user profiles are stored in plaintext). Unless this has changed, I don't find it viable for any app that values user privacy.

Even federated networks eventually gravitate towards a large degree of centralization once the tech becomes mainstream (see email). If any servers are used at all in a privacy-minded app, they need to be zero-knowledge as far as I'm concerned, for actual user data at the very least, if not metadata as well (I realize the latter is an unsolved problem).

How does Actor handle user data and message storage?

Re:Open Sourcers Race to Build Better Versions of Slack by Arathorn on Wednesday March 16, 2016 @05:35PM (#11300864)
Matrix signs all of its messages to avoid homeserver providers tampering with message history. Actual encryption however requires E2E encryption, which is in the wings via our "Olm" Axolotl implementation (http://matrix.org/git/olm). Progress can be tracked at https://matrix.org/jira/browse/SPEC-162. So I don't think you should reject Matrix on trust issues quite yet (although if you're really worried you should certainly wait for E2E to land).
Re:Open Sourcers Race to Build Better Versions of Slack by lewisl9029 on Wednesday March 16, 2016 @08:09PM (#11301695)
Glad to see there's been progress on this front! The lack of E2E encryption is literally the last thing holding me back from building apps with Matrix. Can't wait to give it a try! =)
Re:Open Sourcers Race to Build Better Versions of Slack by zanny on Thursday March 17, 2016 @12:07AM (#11302510)
Axolotl has a well defined message structure using established crypto functions. All of them, for example, except Curve25519 are in Crypto++. You can probably fairly easily bake your own, albeit I'd use the olm from Matrix and just make theirs work well first.
Re:Open Sourcers Race to Build Better Versions of Slack by ar0 on Wednesday March 16, 2016 @07:54PM (#11301620)
But isn't the same true for Slack and its "open source competitors" mentioned in the Wired peace?

At least that's how I read Slack's security page (https://slack.com/security-practices) that only talks about "Data Encryption In-Transit" (i.e. server-to-client) and Mattermost's About page (https://about.mattermost.com/) that also talks (only) about encrypted "client-server data transmission".

I do agree that end-to-end encryption is important, but at least Matrix is aware of this and they are working on it (https://matrix.org/jira/browse/SPEC-162).

Re:Open Sourcers Race to Build Better Versions of Slack by lewisl9029 on Wednesday March 16, 2016 @08:21PM (#11301759)
Not sure if any were mentioned in the piece, but there are plenty of zero knowledge E2E encrypted chat clients out there. None of the ones I'm aware of are also federated like Matrix is though, and I suspect the federated nature of Matrix probably makes E2E encryption a lot more complicated to implement than I can ever imagine.

I definitely appreciate that the Matrix team is aware of and actively working on E2E encryption, but it doesn't change my stance that Matrix is not a feasible building block for privacy-minded apps until that work is complete.

Re:Open Sourcers Race to Build Better Versions of Slack by Arathorn on Wednesday March 16, 2016 @09:17PM (#11301979)
Agreed that E2E is mandatory for privacy. The federated nature of Matrix doesn't make it any harder to implement though - after all, TextSecure/Signal has basic federation too. The one subtlety we have is that we want to give the users the option of selecting between PFS and replayable epochs of history on a per-room basis - the latter making it easier to sync history between devices; the former being for the privacy paranoid.

Now, a more interesting privacy concern is that Matrix doesn't and can't protect metadata currently - see https://matrix.org/~matthew/2015-06-26%20Matrix%20Jardin%20E... for details. So if you want both encrypted contents as well as obfuscated metadata, go check out Ricochet or Vuvuzela or Pond or something for now :)

Re:Open Sourcers Race to Build Better Versions of Slack by lewisl9029 on Thursday March 17, 2016 @12:58AM (#11302664)
Yes metadata is probably going to remain an unsolved problem for Matrix because it's not fully distributed, but I agree with the tradeoff Matrix is making here.

For now, building a great federated chat protocol that allows users to fully control their own data should take priority over just about everything else, because once users are in control of their own data, they're no longer locked to a single platform. So if another protocol comes along that does everything Matrix gets right and adds metadata protection, users can simply export their data and import it to the next great app/protocol (which is hopefully also federated). Until we have a great federated protocol that everybody is using, the friction associated with moving between chat apps/protocols is going to continuously inhibit real innovation in the messaging apps space by favoring apps that stand on the strength of their network effects over apps that stand on their own merits.

On the topic of fully distributed apps, I really like what replikativ is doing in this space:

https://github.com/replikativ/replikativ

It's a CRDT-based data replication library for building fully distributed applications. They envision replikativ being used as an open data exchange whereby users fully control their own data, and can specify any number of applications to have access to that data, instead of the status quo where user data is kept in closed, per-application silos. Their README has a much more comprehensive overview of the library and their vision.

Re:Open Sourcers Race to Build Better Versions of Slack by Spakman on Thursday March 17, 2016 @07:30AM (#11303703)
Have you heard much about the Signal server federation? It's been more than a year since it was mentioned (that I've seen) by the Open Whisper Systems folks.

I'm hoping it's still on some sort of roadmap.

Re:Open Sourcers Race to Build Better Versions of Slack by Arathorn on Thursday March 17, 2016 @10:32AM (#11304543)
I'm not sure it's a huge priority there currently, but we don't have any internal knowledge. We asked to federate Signal with Matrix (which is a concrete possibility given we share the same E2E spec) and were politely turned down, as they'd prefer Signal to be a silo whose security they control rather than risking interop with 3rd parties. Hopefully they will change their mind in future once Matrix's E2E is more proven :)
Re:Open Sourcers Race to Build Better Versions of Slack by ex3ndr on Wednesday March 16, 2016 @05:28PM (#11300820)
We have two options - end-to-end (experimental) and only client-to-server encryption. We are looking at ZeroDB to make it end-to-end by default and searchable. We had e2e default encryption but people start to scream after missing their phones and losing all theirs messages :(

Also we had Tor-enabled client, but Tor is not suitable for mobile apps.

Re:Open Sourcers Race to Build Better Versions of Slack by lewisl9029 on Wednesday March 16, 2016 @08:35PM (#11301821)
The possible integration with ZeroDB definitely sounds interesting! Really hoping that works out for you guys.
Re:Open Sourcers Race to Build Better Versions of Slack by dorfsmay on Wednesday March 16, 2016 @05:40PM (#11300891)
There's been some effort towards integration between SIP and XMPP:

https://tools.ietf.org/html/rfc7247

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @05:12PM (#11300719)
Yeah, I heard about it. It looks very promising, good luck.
Re:Open Sourcers Race to Build Better Versions of Slack by hobarrera on Saturday March 19, 2016 @10:35AM (#11318685)
It's not quite clear how actor.im federates. The "username" is my email, but how does my email map to the server I used to log in?

How do other servers know how to reach me?

Re:Open Sourcers Race to Build Better Versions of Slack by chei0aiV on Thursday March 17, 2016 @04:22AM (#11303130)
Can you make an open source Telepathy backend so I can use my existing IM client to connect to your network?

http://telepathy.freedesktop.org/

Re:Open Sourcers Race to Build Better Versions of Slack by dandelany on Wednesday March 16, 2016 @05:37PM (#11300877)
You're certainly right that the UX of most IRC clients is seriously lacking compared to eg. Slack. But one thing I've always wondered - is there any reason that IRC couldn't be used as the underlying protocol for these Slack-esque apps? There's no reason why the protocol itself has to handle UX features like emoji/embedding/etc. - just send a text token over the wire and have the client interpret it as a specially displayed element. The only thing it may not handle well is file uploads... but even then, I seem to recall IRC having built-in support for sending files?
Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @05:56PM (#11300992)
> is there any reason that IRC couldn't be used as the underlying protocol for these Slack-esque apps

The IRC protocol is old. It predates HTTP. Some things are possible in theory, but pointless if no client supports them.

Look into ircv3 (http://ircv3.net/), they're writing new specs for the IRC protocol to build exactly what you're talking about. It's very promising, although IMHO too little, too late - Matrix looks like it's going to take the crown eventually.

Re:Open Sourcers Race to Build Better Versions of Slack by haroldp on Wednesday March 16, 2016 @08:09PM (#11301698)
I'm sure that I'm over simplifying, but can't you just put tokens or markdown right in the chat for those sort of things?

If you have an old, flat IRC client, it's ugly but you can figure it out. If you have a supported client you get a nice UI?

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @08:41PM (#11301848)
No amount of markdown makes up for the lack of offline messages.
Re:Open Sourcers Race to Build Better Versions of Slack by caf on Wednesday March 16, 2016 @08:54PM (#11301896)
Offline messages is a server issue, though - in principle there's no reason why you couldn't have a server that remembers offline messages and passes them to the client when it reconnects, framing them as regular IRC messages.
Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @09:46PM (#11302078)
There is, actually. IRC has no concept of timestamps. IRC has no concept of foreign metadata, even, so shoehorning isn't even possible without looking horrible.

IRCv3 fixes all that.

Re:Open Sourcers Race to Build Better Versions of Slack by caf on Wednesday March 16, 2016 @10:14PM (#11302163)
Prefixing a date/timestamp unless the client has asked for IRCv3 timestamps is reasonable.
Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Thursday March 17, 2016 @09:47AM (#11304277)
And now, you just broke essentially all clients that already support channel logs by duplicating anything they already had.

This is a problem bouncers are already solving best they can, and it still sucks. It's not good enough to compete with slack.

Re:Open Sourcers Race to Build Better Versions of Slack by caf on Thursday March 17, 2016 @07:29PM (#11308519)
Channel logs is a marginal, client issue, not a protocol issue. Does anyone do client-side logging on any of these other services that do server-side message history?

I didn't say anything about competing with slack.

Re:Open Sourcers Race to Build Better Versions of Slack by Sanddancer on Thursday March 17, 2016 @02:22AM (#11302855)
Yep. This is how slack does it, least for their xmpp gateway. I imagine that their irc gateway is similar.
Re:Open Sourcers Race to Build Better Versions of Slack by bphogan on Wednesday March 16, 2016 @11:24PM (#11302388)
ZNC handles this nicely.

The tech is all there. It just sucks to set it up yourself and should just work out of the box.

Re:Open Sourcers Race to Build Better Versions of Slack by haroldp on Wednesday March 16, 2016 @09:36PM (#11302050)
That is 100% true. But I use IRC and have offline messages. It's IRC, so I had to cobble something together to make it happen, and the UI is ass, but... I have that within the IRC protocol.
Re:Open Sourcers Race to Build Better Versions of Slack by jessegreathouse on Wednesday March 16, 2016 @10:50PM (#11302286)
We already have offline messages, it's called email.

"Oh, Johnson isn't online? better send him an email. /message acmebot email [email protected] Get your ass to mars!" chat bot sees the command and sends him the email.

Sorry if I'm coming off as cynical it's just that I see a trend of the entire community trying to fix problems that aren't real problems.

Re:Open Sourcers Race to Build Better Versions of Slack by dwaltrip on Thursday March 17, 2016 @01:34AM (#11302758)
Chat is not e-mail. It's chat. It has a much different feel to it. One brings a different mental context to e-mail than they do to chat. It is accessed differently, and the use patterns are very different. Hence, solving "offline chat" by sending an e-mail... this is not a real solution.

You could argue that some of the characteristics we find in chat are not useful, counter productive, etc. And there is probably some truth to that. But that is a different argument than what you made.

Re:Open Sourcers Race to Build Better Versions of Slack by dingo_bat on Thursday March 17, 2016 @01:07AM (#11302689)
Your "solution" does not work well with people on mobile. People on 2G/3G/LTE can alternate rapidly between online and offline. This kind of problem is easily solved by the chat server behaving like a store-and-forward device. Which is exactly what email is. But nobody likes to converse entirely using emails.

Chats have an important place because they allow us to say small things, quickly. What you suggest will result in no viable chat for mobiles, only emails. We can probably make a chat app that looks and feels like whatsapp but actually sends an email for every message. Dunno what problems this might have.

Re:Open Sourcers Race to Build Better Versions of Slack by debaserab2 on Thursday March 17, 2016 @02:21AM (#11302851)
That's not really a protocol level solution. Sure, it works, if you know how to do it - but you don't automatically gain that knowledge "out of the box", you have to learn it. It is a real problem to anyone trying it out that doesn't understand why they can't send a message to someone who is offline.
Re:Open Sourcers Race to Build Better Versions of Slack by untog on Wednesday March 16, 2016 @06:33PM (#11301189)
Also mobile notifications - that's a huge benefit of Slack and an IRC app couldn't (as far as I can see) provide them without maintaining a constant connection.
Re:Open Sourcers Race to Build Better Versions of Slack by maaku on Wednesday March 16, 2016 @05:54PM (#11300985)
These services are really replacements for IRC bouncers, which are not as well standardized (but should be).
Re:Open Sourcers Race to Build Better Versions of Slack by strong_code on Wednesday March 16, 2016 @05:45PM (#11300930)
It has DCC protocol built in, but it isn't nearly as robust as most would like for it to be.
Re:Open Sourcers Race to Build Better Versions of Slack by qznc on Thursday March 17, 2016 @07:33AM (#11303712)
I might be alone here, but I don't see the problem. I also do not believe that federation or interop would be a good feature for group chats.

You want to pick and choose your UI? That is an anti-feature. One user sends an image and the other cannot display it because he uses a terminal client. One user sends an emoji, another client only displays a box, because he has no glyph in his font. This is already a problem with XMPP. It has lots of nice features, but each requires server and client to support it. There is no "Full XMPP" label.

And since you prominently talk about the protocol: XMPP probably has all the features one needs. Why hasn't it won yet? Because the complexity of all server-client combinations makes a good user experience too hard.

What is the problem with using different clients/networks for different groups? Why do you want to mix work and private groups into the same app? What does matter (especially for web apps) is a unified authentication scheme (Mozilla Persona would be my favorite), so I can quickly switch between them and do not need lots of accounts. Thus: Federated accounts, but diverse chat solutions.

(disclaimer: I slightly exaggerated to play devil's advocate)

Re:Open Sourcers Race to Build Better Versions of Slack by slang800 on Thursday March 17, 2016 @03:43PM (#11307010)
"One user sends an image and the other cannot display it" deals with features between two clients... I fail to see how this would be a more general issue with allowing different UIs. Especially since most chat clients do support the same features.

That argument is like saying we should only have one HTML browser because text-based browsers like w3m could exist.

And "one user sends an emoji, another client only displays a box, because he has no glyph in his font" is an issue between different operating systems. Even in a non-federated system like Slack I used to get emojis from people using iOS8 that don't display properly.

Re:Open Sourcers Race to Build Better Versions of Slack by andrey_utkin on Wednesday March 16, 2016 @05:19PM (#11300763)
> open source multi-protocol group chat gateway

What about Spectrum http://spectrum.im/ ? I am successfully using Spectrum as gateway to Skype (except groupchats ATM, some bugfixing pending) and IRC from my XMPP client apps.

What about XMPP in the whole? It looked depressing few years ago, but times have changed, I can help you to catch up with nice novelties.

Re:Open Sourcers Race to Build Better Versions of Slack by edhelas on Thursday March 17, 2016 @05:43AM (#11303404)
XMPP has changed yes. And we now have some nice clients that are compatible together and stable servers with a bunch of nice features (multi-devices sync, history management, ACK, rich messages, messages edition, encryption, battery management, push notifications…).

I'm personally working on Movim, a nice-looking "Social-IM" client fully based on XMPP (https://movim.eu/). The main goal of the project is to show that we have now all the tools to build a decentralized, standard and universal IM (and also social !) network without reinventing the wheel again and again.

You can have a look at our latest release-note as well https://pod.movim.eu/?node/pubsub.movim.eu/Movim/f5f883e3122... (this blog post has been published/edited using Movim and XMPP ;)).

And Movim is also compatible with Spectrum to offer interconnection with other networks ;)

Re:Open Sourcers Race to Build Better Versions of Slack by detaro on Thursday March 17, 2016 @07:03AM (#11303632)
> And we now have some nice clients that are compatible together and stable servers with a bunch of nice features (multi-devices sync, history management, ACK, rich messages, messages edition, encryption, battery management, push notifications…).

Can you recommend a few? I haven't looked into XMPP in detail for a while, because the clients I've been using are "good enough and I didn't bother with mobile), but it seems like I'm missing out on good things ;)

Re:Open Sourcers Race to Build Better Versions of Slack by edhelas on Thursday March 17, 2016 @08:48AM (#11303987)
Well I can recommend Conversations for Android https://play.google.com/store/apps/details?id=eu.siacs.conve... which have some really nice features and is fitting well with Movim.

For the desktop there is no real "modern" client yet, unfortunately. But I've ported Movim using Electron :) There is already a Ubuntu/Debian version and the package is here https://github.com/edhelas/movim_electron.

Re:Open Sourcers Race to Build Better Versions of Slack by andrey_utkin on Thursday March 17, 2016 @09:53AM (#11304317)
For desktop, I've heard of Adium for Mac. For Linux, the one with most decent features support is Gajim, but it is far from being perfect.
Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @06:21PM (#11301130)
This is the first time I hear about Spectrum, actually.
Re:Open Sourcers Race to Build Better Versions of Slack by bphogan on Wednesday March 16, 2016 @11:21PM (#11302382)
Edit: looks like someone asked a similar question below.

I'm curious though.

What is it about IRC that makes it not a good choice? It seems to me that the biggest issue non-nerds care about is the user experience. And I feel that if people wanted to, that could be fixed. And if we really wanted to, we could extend IRC to do what we want and deal with its shortcomings.

I use a nice web gateway (kiwi) on top of an IRC channel for my student office hours and it's easy. They just supply a username and log in. No software to install, the UI looks nice, and students don't have to sign up for anything.

If I had the resources and the money, I'd focus on a good modern IRC client and server.

Am I wrong?

Re:Open Sourcers Race to Build Better Versions of Slack by _b8r0 on Wednesday March 16, 2016 @11:43PM (#11302440)
> Am I wrong?

Yes and no. Setting up an IRC server is hard. Pretty much all IRC servers use arcane settings and configuration formats. I run an Inspircd and Hybrid servers, and for people who aren't used to it, it's really daunting compared to setting up something like prosody.

Services on top like Kiwi are nice. I run Shout IRC which is beautiful and slack-like, but again requires passenger and node to work, so lots of moving parts just for a web-ui.

Having said that, all of this is perfectly doable, it's just not so easy if you're used to dealing with something lightweight and a little php on top.

Re:Open Sourcers Race to Build Better Versions of Slack by bphogan on Thursday March 17, 2016 @12:07AM (#11302512)
Yea. And isn't it funny though in an age of Docker and pre-made VMs that we still configure stuff?

Seems like a quick script would set all this up for us :D

Re:Open Sourcers Race to Build Better Versions of Slack by Sanddancer on Thursday March 17, 2016 @02:24AM (#11302859)
A lot of ircds disgustingly put some config options in at compile time, including things like the servername. A friend of mine had her irc server give the generic name for her ircd for several months because she couldn't be arsed to spend the time to recompile and reconfigure.
Re:Open Sourcers Race to Build Better Versions of Slack by bphogan on Thursday March 17, 2016 @09:53AM (#11304313)
Couldn't you just make sure that puppet/chef/whatever runs the right config options? Maybe a script that asks you for values up front and plugs them in?
Re:Open Sourcers Race to Build Better Versions of Slack by foolfoolz on Wednesday March 16, 2016 @11:27PM (#11302399)
i think the #1 value add from slack/hipchat is that it's hosted for you. we spend thousands of man-hours each month and year keeping our servers up. uptime is hard. chat is not hard. historical chat is not hard. when chat is down, your business is at risk. there's other services at my company we pay for that aren't very complicated - we just don't want to host it. paying $5/user/mo for chat that is always up is easier than keeping it up yourself.

your use case is very different, and i wouldn't expect any pay-for-chat service to make any sense for it

Re:Open Sourcers Race to Build Better Versions of Slack by bphogan on Thursday March 17, 2016 @12:10AM (#11302519)
Makes sense.

Although I take issue with "when chat is down your business is at risk."

Pick up a phone. Use Skype. Send email.

Business was done for thousands of years before chat apps. :D

Re:Open Sourcers Race to Build Better Versions of Slack by thenomad on Thursday March 17, 2016 @06:25AM (#11303536)
That works less well if, for some reason, your business is significantly built around chat (Live Chat for sales, for example).

It's perfectly possible to pivot your sales funnel, but it's not something you can do in two hours whilst the chat server's down.

Re:Open Sourcers Race to Build Better Versions of Slack by bphogan on Thursday March 17, 2016 @09:50AM (#11304294)
So you plan for it. We just had this whole discussion on HN a few weeks ago when Mandrill changed and everyone was scrambling.

If you rely on a third party for business, you need to plan for it.

The rule always applies: You will pay now or you will pay later.

Re:Open Sourcers Race to Build Better Versions of Slack by david_ar on Wednesday March 16, 2016 @11:09PM (#11302345)
Note that rocket chat can federate [1] over matrix [2], and I believe mattermost can as well.

[1] https://matrix.org/docs/projects/as/rocket-chat-federation.h... [2] https://matrix.org

Re:Open Sourcers Race to Build Better Versions of Slack by creshal on Wednesday March 16, 2016 @05:21PM (#11300779)
> Some sort of open protocol.

XMPP seems to be rather dead, for various reasons, and no real contender for a replacement has sprung up. I guess there always is, and always will be IRC…

Re:Open Sourcers Race to Build Better Versions of Slack by Zash on Wednesday March 16, 2016 @06:15PM (#11301094)
I use XMPP every day. All my friends are on XMPP. What makes you think XMPP is dead? What are "various reasons"?
Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Wednesday March 16, 2016 @07:04PM (#11301366)
Having used and developed with XMPP day in and out for the past 9 years I'd have to agree with the sentiments towards XMPP being on life support.

It's a real damn shame because XMPP is pretty awesome. It's open for one, can be made super secure with OTR (client-2-client encrypted chats), has numerous protocols and implementations for stuff as diverse as message archiving, Audio/Video signalling, file transfer, chat (of course), presence and an absolute boat load of other stuff: http://xmpp.org/extensions/index.html.

There are two big issues as far as I can see with XMPP which really hurt it:

1. XMPP was incredibly poor at handling high latency, intermittent network connections (like cellular). Connections would time out, reconnections would take ages and all in all it really hammered battery life. As a consequence mobile app developers moved very quickly onto REST/JSON (yes there was XEP-0198 - Stream Management - but implementations of it were thin on the ground and even then it wasn't perfect).

2. There just isn't any interest amongst new developers or newer projects at least in an XML based chat protocol when REST/JSON is so attractive. Anyone who has had to write a god damned XML parser knows what I'm talking about.

Things are a whole lot better now. XMPP has fixed a lot of session and battery life issues since then but to be honest I'm just not feeling it when I go to XMPP meetups. Feel like I'm attending a wake for an old friend and it really is sad because if there was ever gonna be one protocol to rule them all XMPP was far and away the best candidate.

Who knows, a couple years from now large companies and governments might finally wake up to the fact that they have 100 different sucky (probably) REST based APIs amongst all their apps which are not remotely compatible and offer poor archiving and retrieval (like they did in the 90s when they forced Microsoft to open up their Office document format). XMPP might come to the fore again then (wouldn't put money on it though). More likely something similar (and not XML based) will take its place.

It's anyone's guess really, suffice to say the present situation does feel like a complete and utter mess and will really start biting us in the ass when compliance asks 'how do I archive/retrieve this pot pourri of data'.

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @08:20PM (#11301755)
Thanks for the insight. It reflects what I often hear from other people close to XMPP.

What are your thoughts on Matrix?

Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Thursday March 17, 2016 @08:53AM (#11304004)
Looks like they're making all the right noises in order to be a viable successor to XMPP. Really does look good, but seems to be in its very early stages.

Reminds me of what the Layer guys were trying to do (at least in their initial mission statement). https://layer.com

Pros of Matrix are that it looks like they have full chat support, are working on adding end to end encryption, and of course have federation which is super important for interoperability.

Only concerns with Matrix are that I don't see any large projects or companies spearheading it. Also seem resource constrained. For example it looks like they planned on rewriting their existing basic server side app in C and then ended up just writing a proxy. So clearly it's still a small project. Also see verbs in their URIs...I'm not super up to date on REST best practices but isn't that a no-no?

Anyways early days - and it's easy to nit pick. Matrix certainly looks compelling enough that I'd be interested in trying it and seeing them succeed. God knows we need something like it and soon.

Re:Open Sourcers Race to Build Better Versions of Slack by Arathorn on Thursday March 17, 2016 @10:29AM (#11304523)
There are quite a few large/huge companies building stuff on Matrix - looking at the copyright statements of the code in github.com/matrix-org reveals a few. Many of them are using it in commercial stuff which hasn't launched yet - and meanwhile Matrix is still in beta, so understandably they don't publicise their activity too loudly about their activity atm.

In terms of resources: a bunch of us are paid by our dayjobs to work on Matrix fulltime, and meanwhile there's a pretty active wider community surrounding the project. However, the 'core' project of building a spec, reference server, reference+glossy web and native ios/android clients, as well as loads of bridges really is a significant amount of work, and we're going as fast as we can. https://www.openhub.net/p/matrixdotorg gives an idea of progress (although only seems to spider a few of the core projects). I wouldn't really call Matrix a 'small' project :)

In an ideal world we'd stop the world and take N months to entirely rewrite Synapse (~40KLOC of rather dense Python/Twisted) to Go/Rust/C++/Java/Node/whatever, but that's N months we don't have right now. It's not really the lack of resource, but the fact that the window is rapidly closing to provide an interoperable fabric between all of these proprietary communication apps (be they open source or commercial licensed), and we want to get a stable solution out there before it's too late and we end up in a world where the silos have won (c.f. social networks).

Hence the pragmatic decision to do an incremental migration from Synapse to Dendron, which is our next-gen homeserver written in Go (not C). Dendron is indeed mainly 'just' a proxy for now, but this is actually incredibly cool as it can act as a Matrix-aware loadbalancer across multiple Synapse backends. The horizontal scalability and HA this gives is a huge deal. Meanwhile, it gives us a way to gradually migrate endpoints from Synapse into Dendron and provide a way to replace the Synapse codebase without having to do a stop-the-world rewrite. Personally, I think this pragmatism is ftw.

In terms of verbs in URIs... we've been pragmatic rather than religious about REST naming conventions; if someone can give concrete reasons to rename endpoints before we freeze the spec, please tell us! Bugs to http://matrix.org/jira/browse/SPEC and PRs to http://github.com/matrix-org/matrix-doc please ;)

And whatever, please do come hang out on #matrix:matrix.org and #matrix-dev:matrix.org eitherway if you want to find out more and help us escape beta!

Re:Open Sourcers Race to Build Better Versions of Slack by coldtea on Wednesday March 16, 2016 @07:43PM (#11301564)
>I use XMPP every day. All my friends are on XMPP. What makes you think XMPP is dead? What are "various reasons"?

Well, it failed to be the preferred solution for its problem domain, and is universally ignored by big players.

Google also killed Talk in favor of Hangouts (and the later doesn't talk to XMPP).

Re:Open Sourcers Race to Build Better Versions of Slack by stephenr on Thursday March 17, 2016 @01:15AM (#11302712)
> Well, it failed to be the preferred solution for its problem domain,

Google Talk, Facebook Chat, Origin (EA), Playstation, Cisco WebEx, WhatsApp all use/used XMPP for instant messaging.

That sounds like it's the preferred solution for realtime messaging, with vendor lock-in and NIH syndrome decisions being the major reasons that it isn't used in even more situations.

> and is universally ignored by big players.

As evidenced by the above list of major deployments, it is absolutely not "universally ignored". The problem (as someone else pointed out) is that for many of the "big players" a federated protocol is not what they want, so they don't allow federation, and don't allow/publicise the use of standard XMPP clients.

> Google also killed Talk in favor of Hangouts (and the later doesn't talk to XMPP).

This is a reason to stop using Google services, not a reason to claim XMPP is ill-suited to the task at hand.

Re:Open Sourcers Race to Build Better Versions of Slack by mikekchar on Thursday March 17, 2016 @12:30AM (#11302578)
The thing is that Google moved from XMPP to something proprietary specifically to avoid federation. The big players don't want federation and any protocol will suffer the same fate unless it becomes popular without reliance on the big players. I think it is fair to say that XMPP isn't any more dead than any other protocol in this respect.
Re:Open Sourcers Race to Build Better Versions of Slack by tottenhm on Thursday March 17, 2016 @03:24AM (#11302983)
An interesting point. There was a similar issue with SMS (federating with big telcos), and Whatsapp (among others) did a nice job of routing around that (by piggybacking on SMS). It's interesting to think about how a genuinely open player might route around that.

For example, suppose you want to send an IM to your friend. You believe in XMPP or some other open/federated protocol, but you don't know which friends use the protocol. Piggyback off email. Note that identities are backed by email. Also, if any identity-providers refuse to join the federation, then route through a trusted fallback service (Trent.com). Pseudocode:

function send_im(toEmailAddr, messageText) { // See if the user's domain already supports OpenChatProto. if (nslookup(toEmailAddr.hostname, 'SRV', 'OpenChatProto')) return OpenChatProto.deliver(toEmailAdrr, messageText);

  // If user's domain doesn't support it, see if the user has
  // registered+authenticated with a trusted third-party.
  var proxyAddr = http.get('https://trent.com/proxyAddr?email='+toEmailAddr + authCode);
  if (proxyAddr) {
    return OpenChatProto.deliver(proxyAddr, messageText);
  }

  // If the user doesn't exist in OpenChatProto, then
  // fallback to email.
  var emailText 
    = messageText
    + "To continue this discussion in real-time chat, go to:"
    + "https://trent.com/setup?email=" + toEmailAddr + authCode
    + "To block messages from this person, go to:"
    + "https://trent.com/optout?email=" + toEmailAddr + senderEmail + authCode
    + "To opt-out of all messages, go to:"
    + "https://trent.com/optout?email=" + toEmailAddr + authCode

  Email.deliver(toEmailAddr, emailText);
}

Note: It's most efficient if members of the federation agree on the identity of trent.com, but it's not required. Any outbound MTA could use a different Trent, but that fragments the brand and identity databases.

This feels pretty obvious, so maybe it's already being done or I'm missing something...

Re:Open Sourcers Race to Build Better Versions of Slack by msbarnett on Wednesday March 16, 2016 @06:31PM (#11301183)
Google's abandonment of it was a big, big blow. 90% of my contacts are no longer available via XMPP.
Re:Open Sourcers Race to Build Better Versions of Slack by creshal on Wednesday March 16, 2016 @06:46PM (#11301273)
All the servers available for it suck. All the clients available for it suck. I know, I'm using it daily too, but I don't see it gaining more traction that it already has ever again.
Re:Open Sourcers Race to Build Better Versions of Slack by pyre on Wednesday March 16, 2016 @07:05PM (#11301375)
What sucks about (e.g.) Adium? I haven't had any major issues with it other than the fact that it doesn't seem to be too actively updated. My biggest complaint is lack of protocol support for other things (e.g. GTalk vs. Hangouts), but that's on libpurple, and the lack of open protocols by the providers.
Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Wednesday March 16, 2016 @07:09PM (#11301403)
Developer interest is on the wane in pretty much every XMPP project out there. That's the main issue.

XMPP is very much alive in large businesses, especially for system to system interop, but as far as client to server goes I don't see a compelling XMPP client - where's voice, video, chat archiving for example? Compare any XMPP client to Skype for instance and they're not even remotely close.

Re:Open Sourcers Race to Build Better Versions of Slack by pyre on Wednesday March 16, 2016 @07:14PM (#11301424)
> chat archiving for example

This requires a central server though. This is more a function of the service providing XMPP to you than the client. If you want the functionality on the client, it presumably requires some sort of interop between client and server that I'm not sure it part of the XMPP spec.

Re:Open Sourcers Race to Build Better Versions of Slack by Qwertious on Wednesday March 16, 2016 @10:57PM (#11302310)
Sounds like XMPP isn't the solution then, only {???,XMPP} is.
Re:Open Sourcers Race to Build Better Versions of Slack by pyre on Thursday March 17, 2016 @12:35AM (#11302601)
Well, extending the spec could be the solution too. What I was trying to convey was more that there will be a lack of such clients until there is either an agreed-upon extension to the standard, or a popular enough fork of the standard. (at least as far as XMPP is concerned)
Re:Open Sourcers Race to Build Better Versions of Slack by creshal on Thursday March 17, 2016 @07:59AM (#11303795)
There is an XMPP extension for it: http://xmpp.org/extensions/xep-0136.html

The problem is, there are far too damn many XMPP extensions and too few of them see any kind of real adoption.

Re:Open Sourcers Race to Build Better Versions of Slack by MattJ100 on Thursday March 17, 2016 @10:45AM (#11304624)
XEP-0136 was pretty complex, and didn't see that much adoption.

XEP-0313 is a modern replacement, focusing on just the things that people actually need. It's implemented in many servers and clients already (see https://www.zash.se/mam.html ). It's still under development, but it's already in active use by many people.

Re:Open Sourcers Race to Build Better Versions of Slack by creshal on Sunday March 20, 2016 @01:00PM (#11323604)
Great, there's even two extensions on it!

This is why nobody wants to touch XMPP with a ten-foot pole.

Re:Open Sourcers Race to Build Better Versions of Slack by davecridland on Monday March 21, 2016 @09:13AM (#11327806)
So either XMPP is bad because it hasn't changed for years and is therefore old, or else XMPP is bad because there's new replacements for stuff that didn't work so well.

Honestly, there's some days I wonder if there's any way to win.

Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Thursday March 17, 2016 @08:11AM (#11303848)
Pretty much this. Openfire for instance only recently had XEP-0136 implemented across the board on the server side. In the meanwhile I couldn't find a single client with decent support for chat archiving (XEP-0136) on the client side. Not one (and I really looked).
Re:Open Sourcers Race to Build Better Versions of Slack by davecridland on Monday March 21, 2016 @09:09AM (#11327792)
What? Openfire has had XEP-0136 for literally years. I think a little over a decade, actually. It's only recently implemented XEP-0313.
Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Tuesday March 22, 2016 @01:16PM (#11338096)
Hi Dave, firstly thanks for all the hard work on Openfire over the years, very much appreciated.

Secondly regarding XEP-0136 support in OF. I did state that Openfire has only recently had 'XEP-0136 implemented across the board'.

Whilst the Open Archive plugin which provided support is a fine piece of work it was an incomplete implementation with only 1-to-1 chats (no group chat support).

I worked on merging the monitoring and open archive plugins to provide support for both MUC and 1-to-1 using XEP-0136. That work was only fully finished a couple years ago in 2013 so think my comment's still accurate.

Re:Open Sourcers Race to Build Better Versions of Slack by taurath on Wednesday March 16, 2016 @10:06PM (#11302136)
Adium has a UX and UI from the 90s, for one.
Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Wednesday March 16, 2016 @07:13PM (#11301422)
btw. very much agree most of the clients are inadequate but if I could humbly pimp Openfire (I'm a dev there): https://github.com/igniterealtime/Openfire

And ejabberd of course (powering WhatsApp): https://github.com/processone/ejabberd

You'll see that on the server side at least XMPP is very well catered for.

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Wednesday March 16, 2016 @08:21PM (#11301763)
What are the advantages of OpenFire over ejabberd/prosody?
Re:Open Sourcers Race to Build Better Versions of Slack by leonroy on Thursday March 17, 2016 @08:08AM (#11303841)
Super easy to use, nice admin pages, great plugin architecture for 3rd party developers to extend its functionality. Means there's a LOT of cool plugins which add video, screen sharing, chat archiving, etc.

Deal breaker against it (depending on your use case) is lack of support for multiple domains. It can only handle one per deployment.

It also does need at least 512MB RAM which rules it out on embedded devices.

Re:Open Sourcers Race to Build Better Versions of Slack by creshal on Thursday March 17, 2016 @04:39AM (#11303199)
I've used both and moved on to prosody. Ejabberd is just too arcane to configure and debug; and openfire is a damn resource hog for no apparent reasons.
Re:Open Sourcers Race to Build Better Versions of Slack by coroutines on Wednesday March 16, 2016 @10:14PM (#11302164)
I remember like 8 years ago my friends and I avoided XMPP because XML + the overhead of XML.

Now we have more bandwidth, and sending XMPP as a server is negligible.

I just would prefer JSON to XML. :-)

Re:Open Sourcers Race to Build Better Versions of Slack by runchberries on Wednesday March 16, 2016 @06:21PM (#11301131)
Its still widely used in large projects, but those projects normally modify the protocol to fit their needs (whatsapp, riot games)
Re:Open Sourcers Race to Build Better Versions of Slack by chei0aiV on Thursday March 17, 2016 @04:18AM (#11303111)
I think Telepathy was ahead of its time. Shame it didn't get much adoption outside of Nokia and is now on life-support:

http://telepathy.freedesktop.org/

Re:Open Sourcers Race to Build Better Versions of Slack by goffi on Thursday March 17, 2016 @12:30PM (#11305439)
You may want to check http://salut-a-toi.org/ then (I'm working on it), we have a similar architecture and it's a very active project, XMPP only though.
Re:Open Sourcers Race to Build Better Versions of Slack by coroutines on Wednesday March 16, 2016 @09:17PM (#11301981)
As someone somewhat related to an IRC project, it's frustrating talking to people who love Slack because most of what they want is something that would be supported in an IRC client - not the IRC protocol.

Semi-related:

IRCv3 is an advance in the right direction, but I wish they would make huge breaking changes now - instead of introducing new standards to manage a migration to "later IRC".

Two things I want to see are IRC encoded as JSON, and TLS-only servers in the standard as a requirement. They added message tags which I think just makes IRC ridiculous to parse - if the objective is to extend IRC and add ancillary data to the message they should just use JSON already. I think it's a bad move to allow some intermittent migration path if we all know the end-game is IRC encoded as JSON. It'd be dirt-simple to autodetect if you're receiving JSON or traditional IRC, too. You don't need a CAP exchange to tell you "oh look, it's gonna be fucking json, m8".

An example a feature IRC would need to change for is the ability to edit messages. Slack lets you go back and correct already-submitted messages. You don't have this ability in IRC - and there are ethical reasons to not support it. Personally I'd be in favor of IRC having a command for editing messages, and then clients keeping the entire history of edited messages - so you can always see the original. It's not like the server can take that data back - it's jsut assumed clients would show only the latest, revised version of a message.

I could also see having the IRC server support a virtual DCC user to upload files to - so then you could emulate file attachments like you have on Slack. This is something that can be abused though - file uploads are a security and DoS concern. Slack can support certain things because the infrastructure is paid for - with IRC networks it's largely a "free" operation.

In the past I've made a "Hookbot" that triggers and POSTs to outside services just like Slack. On some IRC networks there are network-vetted bots you can request in your channel to configure and make use of. You can largely support everything Slack does in existing IRC, we just needs standards to add some coherency and expectations for developers/users.

UX does matter, I completely agree with you - IRC needs to change minimally to support those Slack-like features, though. Clients need to change. Most of the draw of Slack is embedding linked items in the chat for convenient viewing. Even icky Mibbit does that :p

Re:Open Sourcers Race to Build Better Versions of Slack by conanbatt on Thursday March 17, 2016 @07:05AM (#11303638)
If you opened your chat, you would be losing the capacity to own the people in your chat and thus never get fidelization (or monetization).

Think of a bar that lets you bring drinks from other bars. It doesnt have good prospects.

Re:Open Sourcers Race to Build Better Versions of Slack by scrollaway on Thursday March 17, 2016 @09:51AM (#11304296)
Imagine that. All the money TBL could have made had he kept the web closed.
Re:Open Sourcers Race to Build Better Versions of Slack by thescriptkiddie on Wednesday March 16, 2016 @08:24PM (#11301772)
Re:Open Sourcers Race to Build Better Versions of Slack by viraptor on Wednesday March 16, 2016 @08:28PM (#11301784)
SIP is not am IM protocol. There's one crammed on top of it https://en.wikipedia.org/wiki/SIMPLE_%28instant_messaging_pr... but SIP is just a framing for it.
Re:Open Sourcers Race to Build Better Versions of Slack by bazingatossr77 on Wednesday March 16, 2016 @06:12PM (#11301072)
IRC Cloud is based on IRC:

https://www.irccloud.com/

Re:Open Sourcers Race to Build Better Versions of Slack by fiatjaf on Wednesday March 16, 2016 @06:24PM (#11301149)
It's easier to write your own protocol, that's the reason.

Laziness.

Re:Open Sourcers Race to Build Better Versions of Slack by zanny on Thursday March 17, 2016 @12:19AM (#11302545)
I wrote a Matrix client library in Qt about three weeks ago, and it was really braindead simple.

JSON in, JSON out. HTTP codes tell you what response you got. TLS on an https connection for secure communications. WebRTC for voice calls, and you just query the API for the signaling server to use.

Re:Open Sourcers Race to Build Better Versions of Slack by pekk on Wednesday March 16, 2016 @11:08PM (#11302342)
Have you implemented the IRC protocol? It's horrible.
Re:Open Sourcers Race to Build Better Versions of Slack by Negative1 on Thursday March 17, 2016 @12:26AM (#11302564)
re: Open Protocol, isn't XMPP a viable open source standard?
Re:Open Sourcers Race to Build Better Versions of Slack by amelius on Wednesday March 16, 2016 @06:22PM (#11301140)
> A hundred different chat systems mentioned here. None of them compatible with one another.

Obligatory xkcd reference: ... ok, never mind, you probably all know which one I mean.

Re:Open Sourcers Race to Build Better Versions of Slack by nso95 on Thursday March 17, 2016 @04:07AM (#11303078)
Make a good irc client