Re:SQLite is not a toy database
by benbjohnson on Thursday March 25, 2021 @03:32PM
(#26584401)
I'm in the process of adding read replication to Litestream[1] so folks can scale out the read-side of their SQLite applications to multiple nodes (or replicate to edge nodes for low-latency).
Amazing. This was missing for my use-case and I gave up on LiteStream because of it but now it seems I have to revisit!
Wonderful job.
Re:SQLite is not a toy database
by yellowapple on Friday March 26, 2021 @05:05AM
(#26590076)
Neat. I'll have to give that a whirl.
Would this enable any node to be a writer (i.e. would it lock the DB across all nodes)? Or would I have to designate some "master" server with exclusive write access and have any other servers forward write requests to that server?
Re:SQLite is not a toy database
by benbjohnson on Saturday March 27, 2021 @12:10PM
(#26603306)
It would be the latter. You'd need to have a single primary and replicas would need to redirect writes to that node.
> That being said, if your website has crazy "web scale" FAANGesque needs and you're at the point where you need to write your own replicated datastore, using SQLite as a base and building your own replication layer on top of it (or using RQLite and maybe adjusting it for your needs) seems like a reasonable way to go.
Exactly what Bloomberg did with Comdb2: https://github.com/bloomberg/comdb2
Crazy web scale folks flip from Mongo (NoSQL) to Postgres: https://www.theguardian.com/info/2018/nov/30/bye-bye-mongo-h...
Re:SQLite is not a toy database
by anonymousiam on Friday March 26, 2021 @02:23AM
(#26589279)
Great article. It took them 10 months to successfully migrate! No mention of how many man-hours, but probably a lot.
I'd guess the reason to be that people keep hearing things like "don't use SQLite for websites" and thus don't even try.
> Why do you say it works for "small" websites but presumably not large ones?
Not the GP, but the main reason I wouldn't use SQLite for a large website is that SQLite itself doesn't offer much re: failover/replication (i.e. multiple servers, one database), and I haven't used RQLite enough (or at all; I should fix that) to be comfortable with it in production. Because of that, I'm more likely to reach for / recommend PostgreSQL instead.
That being said, if your website has crazy "web scale" FAANGesque needs and you're at the point where you need to write your own replicated datastore, using SQLite as a base and building your own replication layer on top of it (or using RQLite and maybe adjusting it for your needs) seems like a reasonable way to go.