News for hackers. Stuff that matters.
Re:CRDT: Conflict-free replicated data type by canadaduane on Wednesday November 06, 2019 @12:39PM (#21464874)
Several interesting open source projects are working with CRDTs to make state synchronization in distributed systems an easier problem to deal with:

  - Braid HTTP (https://braid.news/)
  - Automerge (https://github.com/automerge/automerge)
  - Gun (https://gun.eco/)
  - Yjs (http://y-js.org/)
  - Noms (https://github.com/attic-labs/noms)
  - DAT (https://dat.foundation/)
Personally, I'm most excited for Braid's effort to bring state sync to HTTP through the IETF process, as well as Automerge's progress in P2P via Hypermerge (and it's star app, Pushpin). I'd also like to see Gun succeed, but have had a hard time getting started due to visually distracting quirks in its documentation.
Re:CRDT: Conflict-free replicated data type by lucidstack on Wednesday November 06, 2019 @01:01PM (#21465079)
Horde [0] is a really interesting Elixir library for distributed, self-healing state using CRDT

[0]: https://github.com/derekkraan/horde

Re:CRDT: Conflict-free replicated data type by qohen on Wednesday November 06, 2019 @10:08PM (#21470051)
Horde is indeed a really interesting Elixir library. It's based on the CRDT library, DeltaCrdt [0], also by Derek Kraan. He discusses DeltaCrdt here [1] and Horde here [2].

BTW, Horde is a distributed supervisor, i.e. analogous to what regular Elixir OTP supervisors do, except that it can restart processes on different cluster nodes. It also provides a distributed process registry.

Daniel Azuma of Google gave a great 39 minute talk at ElixirConf 2018, "Docker and OTP: Friends or Foes?" [3], where he shows Horde (and DeltaCrdt) in action to keep a multi-player tank video game going while the Docker container it's running in gets killed and and the Elixir processes get respawned on another container. The state is persisted across Elixir nodes in other containers in an instance of DeltaCrdt.

(Again, the talk is really good -- if the above sounds interesting, you'll want to check it out [3]).

[0] https://github.com/derekkraan/delta_crdt_ex

[1] https://moosecode.nl/blog/how_deltacrdt_can_help_write_distr...

[2] https://moosecode.nl/blog/introducing_horde

[3] https://www.youtube.com/watch?v=nLApFANtkHs

Re:CRDT: Conflict-free replicated data type by rch on Wednesday November 06, 2019 @01:40PM (#21465570)
Akka also provides some CRDT based distributed data capabilities:

https://doc.akka.io/docs/akka/current/typed/distributed-data...

Re:CRDT: Conflict-free replicated data type by toolz on Wednesday November 06, 2019 @01:11PM (#21465189)
https://lasp-lang.readme.io/docs is another very interesting project that makes heavy use of many different CRDTs
Re:CRDT: Conflict-free replicated data type by lewisl9029 on Wednesday November 06, 2019 @07:39PM (#21469250)
The IPFS folks also have an excellent compilation of resources on CRDTs available here: https://github.com/ipfs/research-CRDT
Re:CRDT: Conflict-free replicated data type by api on Wednesday November 06, 2019 @02:27PM (#21466112)
LF uses a giant all-encompassing DAG as a CRDT and implements subjective name conflict resolution by computing "views" of the DAG based on various weighting constraints: https://github.com/zerotier/lf

It's a bit like Riak or etcd but can be used across organizational and trust boundaries similar to e.g. a cryptocurrency. Anyone can run LF as part of the same network and both data privacy and security are maintained.

Re:CRDT: Conflict-free replicated data type by lichtenberger on Wednesday November 06, 2019 @02:22PM (#21466054)
I think Automerge from Martin Kleppmann is super interesting :-)

However, as he told me a more sophisticated persistence layer is needed. Looking forward how it's emerging

Re:CRDT: Conflict-free replicated data type by fyp on Wednesday November 06, 2019 @04:00PM (#21467212)
Re:CRDT: Conflict-free replicated data type by lichtenberger on Wednesday November 06, 2019 @04:20PM (#21467456)
Yes, I saw them. Also his book is one of the best computer science books I've read and he gave me a really good advise for my own Open Source project, regarding the use of BookKeeper as a distributed log (instead of Kafka) :-)
Re:CRDT: Conflict-free replicated data type by brianzelip on Wednesday November 06, 2019 @01:00PM (#21465075)
Not open source, but also Actual, a personal budgeting app made by James Long, the author of Prettier.

Cool stuff for sure.

Here’s an insightful podcast about Actual (hosted by the author of Tailwind.css), http://www.fullstackradio.com/126

Re:CRDT: Conflict-free replicated data type by rogerclark on Wednesday November 06, 2019 @01:36PM (#21465520)
Gun is also very unstable, and all of the support projects (superpeers etc) are skeletal at best. it's not usable for production applications yet
Re:CRDT: Conflict-free replicated data type by marknadal on Wednesday November 06, 2019 @01:51PM (#21465678)
Author here.

GUN is running in production with 8M monthly users.

On large sites like HackerNoon and with Internet Archive.

What makes you say it is unstable?

If there is anything, please report it and we'll get it fixed.

Edit: Re grandparent: Will try to add "dark mode" to docs, in meanwhile, check out the docs on plain-text GitHub wiki ( https://github.com/amark/gun/wiki/Graph-Guide ) which is where everything pull from anyways.

Re:CRDT: Conflict-free replicated data type by canadaduane on Wednesday November 06, 2019 @03:40PM (#21466976)
Thanks for your gracious reply, Mark. Your humanity-first approach to everything, as well as the the Gun community are what make it so very promising to me.
Re:CRDT: Conflict-free replicated data type by woah on Wednesday November 06, 2019 @04:20PM (#21467465)
Thanks for the great comment, Steve. Comments like this humble and uplift me as I think of all the things that the GUN community will accomplish together.
[dead]
Re:CRDT: Conflict-free replicated data type by woah on Wednesday November 06, 2019 @05:26PM (#21468237)
Sorry, I was being irreverent about the tone of the earlier comment. I guess that sort of frivolity is not what HN is about so I may delete my sarcastic comment.
Re:CRDT: Conflict-free replicated data type by marknadal on Wednesday November 06, 2019 @04:51PM (#21467846)
I wish my sockpuppets had 6K+ karma :O

These people aren't me. (Thanks to them for kind comments, tho!)

I have a couple throw away HN accounts but I do not use them to talk about gun.

Re:CRDT: Conflict-free replicated data type by rogerclark on Wednesday November 06, 2019 @06:54PM (#21468922)
my problems involved superpeers being flaky with respect to storage and connections, which meant that data propagation and persistence was not reliable. this was in August when i was trying to use it for a production application. of course, this is anecdotal, and it's easy to refute something like this by just saying "works over here". so either i try it again and find specific complaints to report back... or not
Re:CRDT: Conflict-free replicated data type by gardnr on Wednesday November 06, 2019 @07:32PM (#21469207)
I have been eyeing gun for a production app. It would be a better story if you could try it again and report back. Do you still have the code laying around?
Re:CRDT: Conflict-free replicated data type by marknadal on Wednesday November 06, 2019 @10:36PM (#21470179)
Yes please.

I use a distributed testing tool to simulate any setup or edge cases we find, so we can fix it.

So this would be useful to replicate.

Also, a big fix was released early October.

Re:CRDT: Conflict-free replicated data type by lelf on Wednesday November 06, 2019 @01:05PM (#21465127)
Re:CRDT: Conflict-free replicated data type by kedean on Wednesday November 06, 2019 @01:12PM (#21465209)
Is riak still actively maintained or supported by anyone? Last I heard Basho just sort of vanished as a company a few years ago.
Re:CRDT: Conflict-free replicated data type by lelf on Wednesday November 06, 2019 @01:24PM (#21465368)
Yes. NHS.uk, Bet365, TI Tokyo. Bet365 bought all the assets (and open sourced them).

Last release was several months ago.

Re:CRDT: Conflict-free replicated data type by bessarabov on Wednesday November 06, 2019 @01:36PM (#21465515)
- ff — A distributed note taker and task manager — https://github.com/ff-notes/ff
Re:CRDT: Conflict-free replicated data type by Rapzid on Wednesday November 06, 2019 @04:35PM (#21467636)
Does anyone know what CRDT/OT library the VSCODE LiveShare extension uses?
Re:CRDT: Conflict-free replicated data type by lewisl9029 on Wednesday November 06, 2019 @07:34PM (#21469229)
Not sure about VSCode's implementation, but Atom's teletype uses this: https://github.com/atom/teletype-crdt
Re:CRDT: Conflict-free replicated data type by Cogitri on Wednesday November 06, 2019 @01:05PM (#21465125)
Xi-editor also uses a CRDT to merge incoming editing events
Re:CRDT: Conflict-free replicated data type by AriaMinaei on Wednesday November 06, 2019 @01:13PM (#21465218)
Xi experimented with using CRDTs to merge state changes produced by plugins running asynchronously on different threads, but they decided that it was a failed experiment.
Re:CRDT: Conflict-free replicated data type by raphlinus on Wednesday November 06, 2019 @01:17PM (#21465274)
I think that's overstating it a bit. What I would say is that the CRDT added a lot of complexity, and we weren't able to encapsulate that complexity into an "engine" as was originally hoped.

Previous HN discussion on the mini-retrospective might be interesting: https://news.ycombinator.com/item?id=19886883

Re:CRDT: Conflict-free replicated data type by AriaMinaei on Wednesday November 06, 2019 @02:23PM (#21466068)
Thanks for the correction! Is it more accurate to say that in Xi, CRDTs ended up leaking too much complexity for the purpose of synchronizing state between async components of the editor (like plugins) that run on the same machine, but not necessarily for the purpose of collaborative editing between users where there is significant latency?
Re:CRDT: Conflict-free replicated data type by raphlinus on Wednesday November 06, 2019 @02:35PM (#21466217)
I think that's fair. We had an experimental collaborative editing prototype running on Fuchsia, but that was never officially part of the scope of the project. So this is all about whether technology developed for multiuser collaboration is helpful for the single user case.

A good example is inserting indentation in response to, say, the "Enter" key. In a synchronous model, this is pretty straightforward. You calculate the indentation as essentially a function of the contents of the buffer and the delta (the insert of the newline), then any other typing gets processed after that. In a CRDT model, the additional typing can be interleaved in any order, and the merge operation has to converge to the correct results. You can think about how bad that is: the user could type a hundred lines of code, complete with complex editing operations including cut and paste, deletions, undo, and then the auto-indent plugin wakes up, calculates the diff, and sends it to the core. Meanwhile, the user has typed another hundred lines, including more cut'n'pasting of the affected lines. This is perhaps unrealistic, but the CRDT model requires it. And I think it is possible to solve it (I wrote some "rope science" articles, then there was more discussion in several issues), but it's a hugely intricate puzzle compared with just applying it synchronously, and the benefit is not worth it. It's quite practical to compute indentation for any realistic source file in under a millisecond, if you've got a nice fast language.