News for hackers. Stuff that matters.
Re:SQLite is not a toy database by TheRealPomax on Sunday March 28, 2021 @12:51PM (#26611860)
Not really, it still works exactly as you should expect: writes are handled atomically, using a separate file, so if you copy the database file "mid-write" you're not actually copying a database in flux, you're copying the database before the write operation gets applied, and SQLite applies the write after the copy lock is gone.

The main issue here is time: because copying temporarily locks the db out of further changes, it will be "out of sync" if you naively believe that performing a write means your next call will see that data, and you bake that assumption into your code. While there's a wait, on modern hardware using SQLite for its intended purposes (any volume of reads, but low volume of writes), that's just not an issue.

There are other problems associated with file-based backup, of course, such as missing out on in-memory data, or corruptions caused by power outages, but those problems only exist if we were to copy a db once. Backups run regularly, and any data we miss out on the current pass, we'll get on the next pass.

Basically: when used for the purpose that SQLite was created for, file copies are a perfectly fine backup strategy that really only shows its limitations when you start to push your project into "SQLite isn't really appropriate here anymore" territory.

(having said that: the fact that VACUUM can be run into a new file is super nice, and everyone should know that it exists)

Re:SQLite is not a toy database by benbjohnson on Sunday March 28, 2021 @09:44PM (#26616848)
SQLite uses advisory locks but the `cp` command won't adhere to those so you can get a copy that contains only some of the writes from a transaction. If you're lucky, you just get some wonky data. If you're unlucky, you can get a corrupted copy.

Transactions are only atomic from the perspective of other transactions. Other processes or commands on the system can see partial state. This applies to both the rollback journal & the WAL modes.

File copies are also covered in their "How to Corrupt a SQLite Database File" web page[1].

[1]: https://www.sqlite.org/howtocorrupt.html#_backup_or_restore_...