Re:Let's build a distributed Postgres proof of concept
by KingOfCoders on Tuesday May 17, 2022 @01:10PM
(#31413617)
Time might be up (saying as a long term PG user) with Litestream adding distributed features to SQLite, local inprocess databases might be the next thing.
[ Parent
| on HN ]
Re:Let's build a distributed Postgres proof of concept
by cromd on Tuesday May 17, 2022 @02:14PM
(#31414449)
Local inprocess databases might be a bigger thing, but won’t they do nothing to help cases where 10 different applications on different machines need to read and write in the same database? I would think the target audiences are very different, i.e. people who are finding non-distributed PG insufficient probably can’t alleviate their problems by switching to SQLite
Re:Let's build a distributed Postgres proof of concept
by KingOfCoders on Tuesday May 17, 2022 @02:56PM
(#31414946)
It depends on your access pattern, if every application is write heavy to the same data set, this won't work. If every application is light on writes and those are mostly local (like for many SaaS companies), this will work fine with the future Litestream that redirect writes to one db and then distributes the data (if you can live with slightly stale data). Also depends on the size of you company. Slack does 300k/s messages, this won't work :-)
Re:Let's build a distributed Postgres proof of concept
by cromd on Tuesday May 17, 2022 @03:24PM
(#31415261)
Oh, maybe I am behind on this “future litestream” that “redirects writes”. I only knew of the ability to sync/backup/restore. Do you have a link to something discussing those other features?
Re:Let's build a distributed Postgres proof of concept
by KingOfCoders on Thursday May 19, 2022 @03:24AM
(#31431721)
"adding feature"
https://fly.io/blog/all-in-on-sqlite-litestream/
"The upcoming release of Litestream will let you live-replicate SQLite directly between databases, which means you can set up a write-leader database with distributed read replicas. Read replicas can catch writes and redirect them to the leader"
Re:Let's build a distributed Postgres proof of concept
by ankraft on Tuesday May 17, 2022 @04:39PM
(#31416022)
It doesn’t redirect writes but you can create read-only replicas: https://github.com/benbjohnson/litestream-read-replica-examp...
Re:Let's build a distributed Postgres proof of concept
by andrewchambers on Tuesday May 17, 2022 @08:55PM
(#31418052)
You can already do this by using rqlite to redirect writes.
Re:Let's build a distributed Postgres proof of concept
by nijave on Tuesday May 17, 2022 @03:09PM
(#31415099)
Litestream allows scale up application architectures whereas Postgres allows scale out architectures (by only scaling up the database portion instead of also scaling up application servers).
It's basically the same thing as monolith vs microservices but extending monolith to the data persistence layer. With horizontally scaling apps being the predominant architecture right now, I don't really see Litestream changing much.
If you're going to horizontally scale Litestream with a multiple writers you're going to end up introducing all the network and synchronization pieces Postgres architectures already have.
Re:Let's build a distributed Postgres proof of concept
by Mertax on Tuesday May 17, 2022 @07:24PM
(#31417426)
I only have experience with Azure's App Service, but their App Service can scale out and still use the same mounted local storage layer that is shared across all app instances. So I've been considering this for multi-tenant applications where each tenant's data can be siloed in it's own SQLite database.
Re:Let's build a distributed Postgres proof of concept
by nijave on Wednesday May 18, 2022 @07:31PM
(#31428907)
I'm curious if you can get that setup to work reliably (and if it's performant when done). It looks like Azure uses SMB to mount a share so Sqlite + SMB caveats apply (generally getting synchronous writes and locks working correctly over a network filesystem). Additional details https://www.sqlite.org/draft/useovernet.html
Re:Let's build a distributed Postgres proof of concept
by SahAssar on Tuesday May 17, 2022 @06:00PM
(#31416802)
Litestream still only does single-master, right? To me it feels like the next step up would be something like foundationdb (true multi-master) with a good psql layer.
Re:Let's build a distributed Postgres proof of concept
by KingOfCoders on Thursday May 19, 2022 @03:26AM
(#31431727)
I know this is controversial, but I think for 99.99% of companies multi-master is a fetish (been there, bought the tshirt). But if you need it, yes Litestream is going to be single master without automatic failover.