News for hackers. Stuff that matters.
Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by NathanFlurry on Monday March 03, 2025 @12:32PM (#43244355)
Author here, happy to answer questions!
Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by robertlagrant on Monday March 03, 2025 @12:54PM (#43244619)
I love the idea of Durable Objects, and have thoughts about it in the health space (although having a single provider of them is probably a bad thing) but cross-cutting questions such as this seem to be quite difficult to achieve.

From the chat logs example in your article: how do you cope with a requirement such as "I as a user want to see all my latest thread activity in one place, across all my chat rooms?"

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by jmull on Monday March 03, 2025 @01:17PM (#43244879)
That's an easy case: write to two places (one for the chat and one for the user's activity).

In this case I think you can let them become inconsistent in the face of, e.g., write errors.

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by NathanFlurry on Monday March 03, 2025 @03:09PM (#43246075)
This.

> In this case I think you can let them become inconsistent in the face of, e.g., write errors.

For devs using CF Durable Objects, people frequently use CF Queues or CF Workflows to ensure that everything is eventually consistent without significant overhead.

It's a similar pattern to what large cos already do at scale with keeping data up to date across multiple partitions with Cassandra/DynamoDB.

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by korkybuchek on Monday March 03, 2025 @01:07PM (#43244745)
> From the chat logs example in your article: how do you cope with a requirement such as "I as a user want to see all my latest thread activity in one place, across all my chat rooms?"

create a copy, for example

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by neilpa on Monday March 03, 2025 @03:31PM (#43246333)
Randomly noticed your post is dated in the future - December 16, 2025
Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by NathanFlurry on Monday March 03, 2025 @07:26PM (#43248586)
Great Scott!
Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by abdullin on Monday March 03, 2025 @12:53PM (#43244605)
Interesting read, thank you!

Do you use any special tools to manage all these separate databases, track performance and debug problems?

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by NathanFlurry on Monday March 03, 2025 @02:48PM (#43245892)
The folks over at StarbaseDB (https://starbasedb.com/) are working on building tools for shareded SQLite.

From the companies I've talked to, most developers using this architecture are building quick scripts to do this in-house. Both Turso and Durable Objects SQLite already a surprising amount of usage that people don't talk about much publicly yet, so I suspect some of this tooling will start to be published in the next year.

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by grounder on Monday March 03, 2025 @06:12PM (#43247982)
In your chat channel example, you have a table for messages, and a table for participants. How do you join the participants in this chat channel database with whichever database the participants are actually defined in, so the application would be able to show participant details (name, avatar, etc.)?
Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by NathanFlurry on Monday March 03, 2025 @07:46PM (#43248751)
The two tables are intended to be part of the same "chat" partition (ie SQLite database). You can join them with a native SQLite query. Seems I should make this more clear.

Cheers

Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by grounder on Tuesday March 04, 2025 @01:38PM (#43258416)
I understand the two tables in your chat example, I think. I'm wondering how you get the rest of the user profile data (name, for example). Is that table in a totally different SQLite database? If so, can you join on that, or do you need to query it separately? Thanks!
Re:SQLite-on-the-Server Is Misunderstood: Better at Hyper-Sc... by NathanFlurry on Tuesday March 04, 2025 @02:02PM (#43258676)
Yep. Most designs I've seen put user profiles under a different table which requires a separate query.