News for hackers. Stuff that matters.
Re:How Convex Works by jitl on Sunday April 14, 2024 @01:02PM (#40032531)
Did yall build all the database implementation components yourself, or are any of them open source / off the shelf? It sounds like Kafka could do the transaction log job, and overall the architecture sounds similar to XTDBv1 which does use Kafka for that component.

https://v1-docs.xtdb.com/concepts/what-is-xtdb/

I’m also curious if you support unique constraints on non-id columns, or if having writer actions close to the database kind of takes that role.

Re:How Convex Works by james_cowling on Sunday April 14, 2024 @02:10PM (#40033083)
Convex cofounder here. Yep the database transaction logic, timestamp assignment, concurrency control, etc is all custom. This is a pretty key requirement in our ability to perform dynamic subscriptions and automatic caching in real time.

Internally the very bottom of the stack on the hosted product is actually just a write ahead log on RDS and in the open source product it's just sqlite. We'll likely eventually build our own durable write ahead log, and have plenty of experience doing so, but this hasn't been needed so far.

Re:How Convex Works by xixixao on Sunday April 14, 2024 @01:10PM (#40032596)
We open-sourced a single-machine version of the backend: https://github.com/get-convex/convex-backend

It doesn’t use Kafka, the transaction logic is custom implemented in Rust.

Convex does not have special support for unique constraints, as they are easy to implement in your own code (like you wrote, the mutations are happening “inside” the database). This follows the philosophy of making it clear what the database primitives are, and hence what performance you can expect from the database.

Of course since your code is all JavaScript/TypeScript, it’s easy to layer in constraints via a library, like in https://labs.convex.dev/convex-ents/schema#unique-fields