News for hackers. Stuff that matters.
Re:Stop building databases by m9t on Friday December 01, 2023 @03:30PM (#38491896)
Offline/local-first based on SQLite seems hot right now. Third one I’m reading about this week. And it sounds good to me!

But how does it compare to ElectricSQL[1] and PowerSync[2]?

[1] https://electric-sql.com/ [2] https://powersync.com/

Re:Stop building databases by carlsverre on Friday December 01, 2023 @03:43PM (#38492085)
Indeed it's a very hot space! So exciting to see all the different approaches.

ElectricSQL and PowerSync are both tackling the very hard problem of partial replication. The idea is to build a general solution which allows for one multi-tenant centralized db to bidirectionally sync only what's needed on the client side - while still supporting optimistic mutations (and all the consistency/conflict stuff that goes along with that).

The downside is complexity. Both require the ability to keep track of precisely the set of data on each client in order to push out changes to only that subset of the overall database. In addition, specifying which subsets of the database state to pull down requires a new DSL and is a new thing to learn (and optimise). That said, I'm stoked they are taking on this extremely hard problem so when SQLSync is ready for partial replication someone will have already figured out the best practices.

SQLSync, on the other hand, is full db sync. So every client will see a consistent view of the entire database. You might immediately wonder if this is a good idea - and for some apps, it's not. But consider a personal finance app. The main goal is cross device sync, cloud backup, offline capable, etc. In this case having the entire db stored on every device is probably what you want. Another example is a document oriented data model, such as Airtable. Each Airtable could be a distinct database, thus leaving it up to the client to manage which tables they care about.

SQLSync has a ton of work to do to make these use cases possible (still very much a prototype), but my initial tests have been extremely promising. Hope this helps!

Re:Stop building databases by goleary on Friday December 01, 2023 @05:00PM (#38493083)
>But consider a personal finance app. The main goal is cross device sync, cloud backup, offline capable, etc. In this case having the entire db stored on every device is probably what you want.

A bit confused by this. If I'm a developer of a PFM, I don't want anything but a single user's financial data synced to their device. This sounds like partial replication to me.

Re:Stop building databases by carlsverre on Friday December 01, 2023 @05:21PM (#38493316)
Precisely. In the SQLSync model - every user would have a private database just for their data. For example, this is how the todo list demo works: https://sqlsync-todo.pages.dev

(Note: currently SQLSync's server tier doesn't support auth, just random 128bit ids. Auth will come as it matures - but don't use this for anything super secure at the moment).

Re:Stop building databases by rococosbasilisk on Friday December 01, 2023 @05:24PM (#38493362)
Phillip from PowerSync here, always good to see more people working on problems in this space.

A few things to clarify:

>one multi-tenant centralized db to bidirectionally sync

PowerSync supports syncing from multiple databases.

>The downside is complexity.

I'd say this is true if you're building a partial replication system yourself. PowerSync gives you a ready-built system that's been proven at scale and therefore lets you avoid most of that complexity.

>SQLSync, on the other hand, is full db sync.

It's just as easy to sync the full db with PowerSync as do partial sync.

Edit: formatting

Re:Stop building databases by carlsverre on Friday December 01, 2023 @05:40PM (#38493545)
Thanks for the clarifying points Phillip. I'm a big fan of PowerSync! Exciting to see you guys go after the partial replication problem.

I've adjusted my comment to be more clear and hopefully more fair. I didn't mean to mis-imply anything about your service.

Re:Stop building databases by rococosbasilisk on Friday December 01, 2023 @05:56PM (#38493733)
No worries Carl, cheers!