News for hackers. Stuff that matters.

kevinjahns

Re:Show HN: Hocuspocus 4 – Yjs collab server on Sunday May 24, 2026 @03:25PM (#48260252)
Yjs has a bunch of ports to other languages (including rust, python, and go). They are maintained in the Y-CRDT org.
Re:Any Open Source projects in need of documentation writer? on Thursday April 09, 2026 @10:06PM (#47712750)
docs.yjs.dev

I'm currently working on a major rewrite and it would be great to start off with great documentation!

It might be interesting to explain new concepts in the field of collaborative editing (e.g. how to attribute content).

Feel free to pm me at: [email protected]

Yjs is about making things easy. It is a good abstraction to make anything collaborative (not everyone can implement something like prosemirror-collab).

I'd take the slight performance overhead any day if I get guaranteed syncs. Network protocols are not as reliable as you think they are. Detecting random drops of messages is hard. At scale, you are going to appreciate the sync guarantees.

prosemirror-collab doesn't give you offline editing either. Because, guess what - if there is no central server you can't edit the same doc from multiple tabs.

I once had a customer that accidentally deleted part of their database containing Yjs docs. Few of his users noticed, because their docs synced through y-indexeddb.

And it's fun. You can Yjs on anything. There is a company that syncs Ydocs through QR codes.

As a generic collab library, it does a very good job. CRDTs really are a fun thing to use. A lot of people feel that way.

If you want to use something else, that's totally fine! Write an article about how great prosemirror-collab is.

This is not a fundamental issue. Node splitting can be represented in CRDTs. It's just really hard to map correctly to ProseMirror as the merge logic is complex and bound to a schema.

I don't blame you for ProseMirror for being how it is. I'm just offering my feedback for your next editor library.

I believe that node-splitting in y-prosemirror can be solved.

But I hope that you also can appreciate that mapping prosemirror to a CRDT structure is a very complex thing to do. Your schema implementation and node-splitting behaviors are extremely hard to map to existing conflict resolution libraries.

Other editors, like Quill, map better to CRDT structures.

This is, of course, not your problem. But if you - or any editor author for that matter - ever end up creating another editor library, I'd love to work with you on adaptability to existing conflict resolution libraries. There is a lot to gain from being compatible to the wider ecosystem.

After all, we shouldn't expect our users to set up a tailor made backend for each single collaborative component.

Encrypted editing apps like Proton Mail Docs wouldn't be possible with your solution.

You misunderstand how the "document replacement" in y-prosemirror works. It's like arguing that React is bad because it performs a complete document replacement on every change. The diffing part makes it fast.

That said, it's not without problems - I acknowledge that. But it's not as bad as you make it sound. You didn't list one concrete case when you had issues with it.

I'm very happy that Nick and I finally found funding to make a rewrite happen. It really did take 6 years to make this happen, because it's hard to find funding for open source projects.

Hi, the author of Yjs here. Thanks, Nick, for chiming in!

As this article is blowing up now, I want to address a few points.

I, too, feel the need for simplicity over overly complex solutions - and I found it in CRDTs. They beautifully allow me to reason about conflicts - so that my users don't have to. Very few people can design a custom conflict resolution algorithm for an application. Yjs is a general-purpose framework that enables you to make EVERYTHING collaborative. That's the goal.

It's fine if you want to explore different solutions. I don't understand the need to put down one framework in favor of another. It doesn't have to be "OT vs CRDT". Hey, if you found something that works for you - great! But let me tell you that neither solution magically makes everything simpler. There is still a lot to learn.

Different solutions to conflict resolution have different tradeoffs. It's unfortunate that the author of the article attributes all complexity to Yjs. It's just that collaborative editing is a very complex problem and requires a lot of attention to detail. In many regards, Yjs has done very well for the larger ecosystem. In other regards there is room for improvement.

The only thing I acknowledge from the article is the criticism about y-prosemirror "replacing the whole document". Unfortunately, the author extrapolated some false assumptions. This is not a performance issue. y-prosemirror runs at 60 fps even on large documents. We leverage ProseMirror's behaviour to do identity checks on the nodes before updating the DOM. However, it's true that this breaks positions for some plugins (e.g. a comment plugin).

Instead of prosemirror positions, we encourage plugins to use Yjs-based positions, which are more accurate in case of conflicts. Marijn talks about this as well [1]. The collab implementation in Prosemirror does not guarantee that positions always converge. That means, comments could end up in different places for different collaborators. This works in most cases, but in some it doesn't - which is one of the reasons why I prefer CRDTs as a framework to think about conflicts.

But as Nick said, we are currently working on a new y-prosemirror binding that works better with existing plugins.

I'm curious about the section "CRDTs are much, much harder to debug" which ironically talks about how hard prosemirror-collab is to debug. You won't find any such bugs in Yjs. The conflict-resolution algorithm is quite simple and has been battle tested. Before every release, Yjs undergoes extensive fuzz testing for hours in simulated scenarios. I'm very happy to show anyone how to debug a CRDT. It requires some background information, but it ultimately is easier.

To address another unfounded claim by the author: I bet OP $1000 that the GC algorithm in Yjs is correct even in offline-editing scenarios. He won't be able to reproduce the issues he is talking about.

[1]: https://marijnhaverbeke.nl/blog/collaborative-editing-cm.htm...

Author of Yjs here. I'm all for faster data structures. But only benchmarking one dimension looks quite fishy to me. A CRDT needs to be adequate at multiple dimensions. At least you should describe the tradeoffs in your article.

The time to insert characters is the least interesting property of a CRDT. It doesn't matter to the user whether a character is inserted within .1ms or .000000001ms. No human can type that fast.

It would be much more interesting to benchmark the time it takes to load a document containing X operations. Yjs & Yrs are pretty performant and conservative on memory here because they don't have to build an index (it's a tradeoff that we took consciously).

When benchmarking it is important to measure the right things and interpret the results somehow so that you can give recommendations when to use your algorithm / implementation. Some things can't be fast/low enough (e.g. time to load a document, time to apply updates, memory consumption, ..) other things only need to be adequate (e.g. time to insert a character into a document).

Unfortunately, a lot of academic papers set a bad trend of only measuring one dimension. Yeah, it's really easy to succeed in one dimension (e.g. memory or insertion-time) and it is very nice click-bait. But that doesn't make your CRDT a viable option in practice.

I maintain a set of benchmarks that tests multiple dimensions [1]. I'd love to receive a PR from you.

[1]: https://github.com/dmonad/crdt-benchmarks

Re:Funding decentralised/local-first applications for GNOME on Saturday April 23, 2022 @08:29AM (#31133698)
Work with us! The y-collective [1] is building the foundational technologies for local-first, collaborative applications based on CRDTs.

[1]: https://opencollective.com/y-collective

Re:Conflict-Free Replicated Data Types (CRDT) on Monday April 11, 2022 @08:54AM (#30988003)
It's interesting to me that people still doubt that CRDTs can be used in practice. The fact is that they are already being used in practice by many companies with millions of users for applications like rich-text editing and state management in 3d applications. See https://github.com/yjs/yjs#who-is-using-yjs

In fact, Yjs - a CRDT implementation- is the most downloaded solution for collaborative applications. Even more than ShareDB (the most popular OT-based solution).

Yjs ~90k/week - https://www.npmjs.com/package/yjs

ShareDB ~13k/week - https://www.npmjs.com/package/yjs

Instead of building another shared-editing solution specialized for Matrix, there could be an API that can be used to store and distribute real-time updates efficiently (probably in the Matrix DAG).

The matrix-crdt works really well. To reduce overloading the Matrix server with many small messages (each single keystroke produces an update message), it stores merged updates in the DAG after a short debounce. The optional WebRTC extension allows you to distribute messages immediately "of the chain", so you don't notice the debounce.

After a time the message-log gets pretty huge. So in matrix-crdt, a random client will eventually store a "snapshots" of the current state in the DAG and removes old entries. This way, new clients don't need to download the huge message-log.

It would be nice if there was a possibility to create a server-component that does the merging.

(Btw, all credit to the above approach goes to Yousef)

Now, there might be a better solution to store CRDT data in the Matrix DAG - the developers probably know best and might be able to expose some hidden API that would make everything even more efficient.

I'm just asking that instead of creating yet another CRDT and integrating it into Matrix, open up this space, provide better APIs, and let others integrate their CRDTs.

> Would it be able to abstract any of the underlying CRDT logic?

Modern shared-editing frameworks don't require you to think about internal logic. They just set some requirements on the ordering of update messages. CRDTs in particular don't care in which order you transmit data, which makes them a very interesting choice in practice.

What I dislike about these attempts is that you will just end up with yet another CRDT implementation that is incompatible with the existing ecosystem (editors, drawing apps, state management, ...).

Instead, I want to encourage you to build an API that others can use to efficiently store shared data. Feel free to ping me if you need input.

- Kevin (Author of Yjs)

I can only answer to the last question. Yjs uses several performance optimizations to produce small documents (both in memory and in the encoded state). Since humans type relatively slow (<60 actions per minute), it is impossible for humans to create a document that has performance problems. I showed this in [1].

Relm [2] even models a 3d world using Yjs. I don't necessarily recommend doing this as 3d applications usually produce a lot more actions per minute than text applications. This required some workarounds and deep knowledge of how Yjs' optimizations work. But it's definitely possible.

[1]: https://blog.kevinjahns.de/are-crdts-suitable-for-shared-edi... [2]: https://www.relm.us/

Re:A state syncing framework based on Vuex and rollback netcode on Thursday December 09, 2021 @08:23AM (#29497028)
This looks similar to SyncedStore which was posted just yesterday: https://news.ycombinator.com/item?id=29483913
Yjs author here. On network failure, the client will automatically reconnect and synchronize with the backend. And yes, you can get an event that notifies you about the current connection status. When syncing with the server, you will only exchange the differences that were created while offline.
Yjs author here. Yjs ships a flexible selective undo-redo manager. You can define one (or several) undo managers that listen to changes that you want to be able to undo&redo.

https://docs.yjs.dev/api/undo-manager

Re:Faster CRDTs: An Adventure in Optimization on Saturday July 31, 2021 @01:32PM (#28020079)
I know that it is hard to comprehend why modern CRDT implementations are fast. But the data confirms that they work great. OT seems to be much simpler, but there are real advantages in using CRDTs. The performance problems have been solved through an efficient representation of the CRDT model.

The gist of the below [1] read is that it is impossible for a human to create a document that Yjs can't handle (even in the worst case scenario). But yes, it handles real-world scenarios particularily well.

The concept of "hidden classes" is super old. It has first been implemented in a fork of smalltalk and then became foundational concept of runtime engines for scripting languages. It is implemented in V8, python, ruby, spidermonkey, ..

Yjs does not assume a "real-world scenario" and it is not optimized for any specific runtime engine. It runs fast in any browser. The benchmarks confirm this. [2]

Yjs is being used in practice by several companies (eg Nimbus Notes with >3 million users) for quite some time now. I'm not aware of any performance problems.

[1]: https://blog.kevinjahns.de/are-crdts-suitable-for-shared-edi... [2]: https://github.com/dmonad/crdt-benchmarks

Yjs is exactly that. It is a simple abstraction for building any kind of collaborative application. It has ready to use solutions for most problems related to this problem space. The selective UndoManager, for example, is generic&configurable and can be reused for all kinds of stuff. It supports many different editors. It supports many different (scalable) backends.

It is much more than state synchronization.

If there is anything missing, then let's work on it. Yjs is extensible and allows for custom features that others can reuse.

Neither of them says that. The xi CRDT was not about shared editing. The second paper just says that CRDTs are not superior per se. But CRDTs are being used in production for text editing and they are performant if implemented correctly (even in js).

The following article shows that:

https://news.ycombinator.com/item?id=24176455

Re:Show HN: Check if a GitHub repo is open to contributions on Saturday January 02, 2021 @04:51PM (#25616902)
My projects get a very discouraging rate of ~60%. That is probably because I reject a lot of PRs created by bots (dependabot, security fixes if they are only to dev dependencies).

Maybe because I merge commits manually.

I remember rejecting only a few PRs.

It seems very discouraging for new contributors.

Re:CRDTs are the future on Thursday October 01, 2020 @11:58AM (#24652403)
- The gorgeous automerge API frontend ported across

I want all of that too. Although I would add Automerge's API on-top of Yjs as an additional layer. It really does play nicely with Web Frameworks like react. Although for building editor bindings I would choose the Yjs types because they allow faster access to the data. Yjs only provides a minimal abstraction around the CRDT. I hope that other people will build nice stores/abstractions around Yjs that make interacting with shared data easier.

- Presence information and markers (this is sort of supported now?)

Presence is handled by a different CRDT (the Awareness object in y-protocols). All providers implement the Awareness protocol. In many cases you want one awareness instance for several documents, so it makes sense to separate them.

Markers is probably what I refer to as relative positions: references to specific positions in the document. It even plays nicely with undo-redo.

- Better support for long lived documents. (I want to be able to pull historical deleted stuff out of a document so it doesn't grow unbounded).

Yjs does this reasonably well. To a point that you can even build 3d applications like relm.us with Yjs. But you are right that more work could be invested in researching even better solutions.

- Support for moving objects (as well as inserting and deleting)

Moving of elements can already be modeled with Yjs [1]. Although the approaches described are not very convenient to use. Automerge implements this feature natively, but I'm a bit hesitant to implement the next best approach because I see many downsides that conflicts with features I plan to implement. There is a discussion thread where I outline a few of the tradeoffs [1].

[1]: https://discuss.yjs.dev/t/moving-elements-in-lists/92

Re:CRDTs are the future on Tuesday September 29, 2020 @10:34AM (#24628305)
What are you missing from Yjs?

I'm sharing the vision with you that things should just work. It literally takes you two lines of code to make any of the supported editors collaborative. The Yjs types are a minimal abstraction over the crdt model. I hope that other people build more abstractions around Yjs (for example a react store that simply syncs).

The thing is.. all of what you are talking about is already possible. A wasm port will eventually bring some performance improvements. But Yjs is already very close to optimal performance and it works in all browsers.

But I really am interested in more feature request. I'm mainly interested in building collaborative apps with CRDTs. CRDTs as a data model is an entirely new concept and we need to get experience on how to build apps with CRDTs. And also we need to find out what is still missing.

So head over to discuss.yjs.dev and share your experience.

Re:Are CRDTs suitable for shared editing? on Monday August 17, 2020 @09:52PM (#24194504)
> I have a strong sense of despair seeing how much time I’ve wasted investing in approaches which won’t feature strongly in the future.

Your comment makes me sad on several levels..

Firstly, ShareJS and OT types was the first open framework to build collaborative applications on the web. Your work inspired me to work on shared editing - I just wanted it to work over WebRTC.

Until 2015, JavaScript engines implemented different garbage collection approaches that wouldn't allow efficient CRDT implementations (as they need to handle millions of objects).

The idea of distributed applications on the web just popped up a couple of years ago. WebRTC didn't even exist when you started ShareJS. Even Websockets were still a bit experimental. OT was the right technology at the time.

Lastly, I'd love for you to try out Yjs. The current JavaScript implementation handles 2.5 million operations / second. A Rust/C implementation would surely handle more than that as it is not limited by automatic garbage collection.

You can compare Automerge's current performance branch with Yjs' current implementation in [1]. To be fair, their implementation is not finished. If I understood correctly, they want to load the compressed format directly into memory. I played with this idea a few years ago and represented the Yjs model in an ArrayBuffer. This approach will improve load-time, but the performance will be intolerable in other aspects (e.g. when applying document updates, transforming the document to a String/JSON, or when computing diffs). The performance of Yjs is the result of more than five years research. I still have a couple of ideas to improve performance significantly. Although, my article clearly shows that it is definitely good enough now.

[1]: https://github.com/dmonad/crdt-benchmarks/pull/4

Re:Are CRDTs suitable for shared editing? on Monday August 17, 2020 @01:26PM (#24189435)
Based on the [B4] benchmark, we can predict the size Yjs document representing the complete editing history of the Linux kernel (probably the largest Git repo ever created): 864 MB. The size of the Git repository is currently 1.1 GB. So Yjs has better encoding.

If you would load the the Yjs document containing the editing history of the Linux kernel in-memory, you would use about 13.8 GB of memory. Of course, you wouldn't write the complete Linux kernel in a single file. As of 2011, the project consisted of ~37,000 files. If you represent each file as a separate Yjs document, you would use, on average, just a few kilobytes to load a single file.

The editing history of the Linux kernel is a very interesting benchmark resource. Maybe I will add it to crdt-benchmarks.

Re:Are CRDTs suitable for shared editing? on Monday August 17, 2020 @12:53PM (#24189041)
I think we can agree on that. CRDT vs OT is a complex topic with lots of pros and cons for either side.
Re:Are CRDTs suitable for shared editing? on Monday August 17, 2020 @11:57AM (#24188421)
Great! :)
Re:Are CRDTs suitable for shared editing? on Monday August 17, 2020 @11:39AM (#24188261)
Since there is no concurrency in contact tracing (only you will manipulate your own data), a CRDT might be an unnecessary overhead. The German Corona-Warn-App is a decentralized approach for contact tracing. It's awesome, you should check it out: https://github.com/corona-warn-app
Re:Are CRDTs suitable for shared editing? on Monday August 17, 2020 @11:29AM (#24188168)
OT comes with very different problems when it comes to representing tables. Structured content (tables, tables containing tables, tables containing quotes containing tables, ..) is much better represented using CRDTs.

And another note, Yjs completely supports the OT rich-text type (i.e. delta format [1]). This is how the Quill editor was made collaborative.

So Yjs is at least as powerful as OT. If you want, you can still base your application around OT deltas, but Yjs is IMO much easier to use.

[1]: https://quilljs.com/docs/delta/

Older »