News for hackers. Stuff that matters.
Re:Siren Call of SQLite on the Server by crazygringo on Tuesday February 18, 2025 @10:23AM (#43090537)
As long as the connections are from the same machine (local filesystem), it's fine.

Reads are concurrent, writes block reads (and other writes) while they write. (There's also something called WAL mode that enables reads during a write, which uses multiple files for each database.)

Re:Siren Call of SQLite on the Server by kbolino on Tuesday February 18, 2025 @10:45AM (#43090827)
They don't just get blocked, they error out. This is a well defined error condition, so you can detect it easily, but it's not going to happen if you stick to the one connection rule. This means you have to intentionally think about handling it everywhere if you want to use another connection anywhere.

Edit: looks like you can set up blocking on the client using pragma busy_timeout [1] so the above is only true in default configuration

[1]: https://www.sqlite.org/pragma.html#pragma_busy_timeout

Re:Siren Call of SQLite on the Server by znpy on Tuesday February 18, 2025 @06:27PM (#43096467)
> This means you have to intentionally think about handling it everywhere if you want to use another connection anywhere.

so things like mutexes (one process, multiple threads) or mutexes over shared memory (multiple processes, multiple threads) ?

sounds complicated (and error-prone).

but at the end of the day i guess that kind of juggling has to be done somewhere, either a database server does it for you or you do it yourself in the database client.

i wouldn't mind sqlite on the server, but i'm not sure I would want to trade zero-downtime-deployments (spin up new version, move load from old to new, spin down old version) for single-file ease of operations.

Re:Siren Call of SQLite on the Server by crazygringo on Tuesday February 18, 2025 @07:36PM (#43097062)
You don't have to really think about it anywhere.

It's handled by the file lock. You don't need to track anything.

Yes there is a timeout value that will return an error if the database file hasn't unlocked in time. In Python the default timeout is 5 seconds, which is plenty if your queries execute on the order of milliseconds or tens of milliseconds.

Re:Siren Call of SQLite on the Server by hruk on Tuesday February 18, 2025 @07:26PM (#43096983)
We do zero-downtime deployments with a single Docker volume containing the db. Spin up a container running the new code, wait til it's healthy, then kill the old container.
Re:Siren Call of SQLite on the Server by wmanley on Tuesday February 18, 2025 @12:41PM (#43092688)
You definitely want WAL mode (unless you’re accessing SQLite over NFS). It’s only not the default for backwards compatibility. I’d enable:

PRAGMA journal_mode = WAL; PRAGMA foreign_keys = 1; PRAGMA mmap_size = 1099511627776;

Unless you know a reason that you can’t (like NFS or using a 32-bit cpu).

Re:Siren Call of SQLite on the Server by znpy on Tuesday February 18, 2025 @06:28PM (#43096473)
> Unless you know a reason that you can’t (like NFS or using a 32-bit cpu).

interesting, why there's that limitation when nfs and a 32-bit cpu are involved?

Re:Siren Call of SQLite on the Server by wmanley on Wednesday February 19, 2025 @08:42AM (#43102045)
WAL mode requires shared memory, so doesn’t work over the network.

mmap_size = 1TiB requires 1TiB of address space per connection. This is fine on 64-bit, unless you have millions of connections. Doesn’t work with 32-bit (4GiB) address space.

You want mmap_size to be greater than the size of your database. If you know your database will be small then you could set it to a smaller number, but you could still become address space constrained with many connections.