Hi! I want to create a web app like Google docs where multiple users can collaborate in real time to edit the document together (using a special link like gdocs generate). I want to save the docs in a MySQL db (no firebase)

My questions are:

1) Since multiple people are working together how does one manage conflicts, i.e. 2 people sending different edits simultaneously.

2) If one clients gets disconnected (4g) and then reconnects later how does it sync the changes it made during it was offline?

I recently watched this RAFT presentation (1) and I think I would need to use something like this?

What other alternatives are viable?

Also can I make it happen using just PHP, Javascript and MySQL?

Thanks

(1) http://thesecretlivesofdata.com/raft/

I made a document editing demo here: https://github.com/fanout/editor

It uses operational transformations ("OT") to manage conflicts, and it saves the data in MySQL. Technically any Django DB backend will work for storage, but the public demo instance uses MySQL.

One of the reasons I made this thing was to show that realtime apps don't need to require heavy frameworks or unusual databases. And it loads super fast.

I don't think you need Raft if you have a central database storing the document. You could also consider using CRDTs instead of OT, which may be more powerful but also more challenging to develop.

https://en.wikipedia.org/wiki/Conflict-free_replicated_data_... https://en.wikipedia.org/wiki/Operational_transformation

ofcourse u will need websocket to sync changes.

Big question! It deserves its own blog post haha.

CRDTs can be the answer. We actually wrote about them recently (https://ably.com/blog/crdts-distributed-data-consistency-cha...), and there is more coming soon as our Chief Of Scientist and his team are researching CRDTs and building demos!

Have a look at https://m-ld.org/ , its a CRDT implementation that's all Javascript. it could help

Looks very promising. Thanks for sharing!

Yup, and it supports Ably too, see https://js.m-ld.org/#ably-remotes!

Matt, Ably co-founder

CRDT solve a part of your problem, and an important consideration is whether or not you want off-line editing. If you don't need off-line editing, then a WebSocket can do it.

I'm actually using my project to build a collaborative IDE (designer like Figma): http://www.adama-lang.org/

I'm going to be launching it as a SaaS soon so people can spin up a new back-end without managing an infrastructure.

What is the right way to handle authentication over web sockets?

As with REST APIs, you'd want to be able to authenticate to a websocket-based API using either basic auth, or a bearer token-based auth scheme. Unfortunately, the browser websocket API doesn't allow you to specify arbitrary headers in the websocket request, so it's typical instead to have credentials supplied via a query param (such as "accessToken" for a bearer token) in the wss request.

> so it's typical instead to have credentials supplied via a query param (such as "accessToken" for a bearer token) in the wss request.

If someone ends up actually doing this in a production system, remember to not to log the accessToken if you're logging full paths/URIs somewhere, as query params usually is a part of that type of logging.

Yup, true, although tokens should be ephemeral so less of a risk. Authenticating inline over the Websocket connection is valid too, but it does expose the socket connections to slightly more surface area of attack i.e. if you pass in a token as a param, the Websocket request can be rejected immediately. If however you authenticate after establishing a Websocket connection, then there is an attack vector where you simply open Websocket connections and never authenticate. Of course timeouts can be used to disconnect rogue actors, but it is a consideration.

Matt, Ably co-founder

It's the same thing as HTTP. Websocket starts off as an HTTP request with cookies, headers etc. Use those just like HTTP to authenticate, and your Websocket server should pass the user data to the websocket object

Don't have access to the headers from JS.

Best solution might be to generate a short-lived one-time-use ticket and pass it in the querystring.

If you make a normal HTTP request first, the server can issue a standard HTTP cookie to the client. That cookie will then be included when the browser makes the websocket request.

However, websockets are not subject to the same-origin policy, so this exposes you to CSRF [1]. To protect against that, you should check the Origin header on the server side.

[1] https://christian-schneider.net/CrossSiteWebSocketHijacking....

Cookies will be forwarded though, or..?

In a Golang app I'm writing now, I have middleware that authorizes requests. Authenticated requests have a header with a JWT token. I have an endpoint for websockets where if an authenticated request comes in (the handshake) that request is then upgraded to a Websocket connection. This is the cleanest authentication implementation I've ever used thus far, and I wasn't able to achieve the same thing in Node when I was using socket.io.

I'm sure there are repercussions to this on the client-side, but I haven't gotten to that point yet. I'm still writing the server and testing it using automated integration tests.

If I understand correctly, websockets is a thin layer on TCP that does buffer data so that the application get the whole message instead of chunks. I recommend using wss to secure the websocket so that it can't be hijacked, then you don't need to send a token in each message and can do an application layer handshake once. Basically the first websock message from the client would be an authentication message with a password, token or what not.

Yeah this is the technique I've also used.

The first websocket message is the original request, which will have the users cookies / headers where your session information / bearer token should live.

It's a good introduction, and it's a good document to introduce the problems induced by WebSockets (which Ably can come in an solve at scale).

I recently wrote about the Woes of Websocket: http://www.adama-lang.org/blog/woe-of-websocket with an errata based on HN feedback: http://www.adama-lang.org/blog/more-websocket-woe

The depth of this topic is very interesting, and I'm excited as I'm building some of the final pieces for my SaaS (which could compete with Ably).

Is Ably playing with WebTransport? If it moves up from its current trial status, how do you see that changing Ably’s core?

We are monitoring it closely and super excited about what WebTransport will provide, which is both a more reliable and in many cases more performant transport. However, much like WebSockets, it's still quite low level and only provides a basic communication protocol. As such, like we have done with SSE, HTTP, Websockets and MQTT, our service focusses on what developers can enable on top of these lower level transports, such as presence, deltas, history of streams, limitless scale and fan-out of data, and the list goes on https://ably.com/platform.

When WebTransport reaches prime time, I'm confident we'll be supporting it.

Why does Klopp love to abuse referees and then play dumb?

> or anything related to Liverpool FC.

Do people still watch football?

What are the viewership numbers for Liverpool FC?

Does Man United still matter?

:-)

Man Utd never mattered. Source: City supporter.

YNWA!