News for hackers. Stuff that matters.
Re:Things that surprised me while running SQLite in production by globular-toast on Tuesday July 04, 2023 @05:15AM (#36584128)
So can you just run Django with SQLite (the default) now, with no special setup? And no data loss? Years ago I tried that when I made a simple web app for a one-off data collection at work. I had maybe 10 users on it and started to encounter concurrency problems. This was about 5 years ago. What's changed since then?
Re:Things that surprised me while running SQLite in production by ilyt on Tuesday July 04, 2023 @06:13AM (#36584602)
maybe they found proper SQLite options

enabling WAL helps a lot as it allows concurrent reads and a write

https://www.sqlite.org/wal.html

Re:Things that surprised me while running SQLite in production by joseferben on Tuesday July 04, 2023 @06:41AM (#36584827)
Works for me, so far no data loss after about 150 migrations across 3 projects. These are the settings:

@receiver(connection_created) def configure_sqlite(sender, connection, *kwargs): if connection.vendor == "sqlite": cursor = connection.cursor() cursor.execute("PRAGMA journal_mode = WAL;") cursor.execute("PRAGMA busy_timeout = 5000;") cursor.execute("PRAGMA synchronous = NORMAL;")

Initially, I was worried that Django might screw up migrations since it's a bit awkward to make certain schema changes in SQLite. Also almost no one seemed to use SQLite with Django in production, so I was a bit worried about how battle tested the SQLite part of the ORM was.

So far so good.

Re:Things that surprised me while running SQLite in production by simonw on Tuesday July 04, 2023 @07:00AM (#36584990)
I wrote this about WAL mode with Django and SQLite last year: https://simonwillison.net/2022/Oct/23/datasette-gunicorn/

Your configuration looks right to me.

Re:Things that surprised me while running SQLite in production by joseferben on Tuesday July 04, 2023 @07:28AM (#36585217)
Thanks for the link.

> I asked Django Fellow Carlton Gibson what it would take to update that advice for 2022.

Glad to see that SQLite is being presented as a valid option in the current version of the docs.

> Even without WAL mode, bumping the SQLite “timeout” option up to 20s solved most of the errors.

Can you say anything about the tradeoffs when increasing the timeout, maybe in the context of Django?

Re:Things that surprised me while running SQLite in production by simonw on Tuesday July 04, 2023 @07:35AM (#36585290)
I've not explored it in any more depth than I wrote about in that article and my issue comments.

My hunch is that if you bump it up to 20s while receiving a huge amount of concurrent traffic you may find yourself running out of available Python workers to serve requests.

Re:Things that surprised me while running SQLite in production by troppl on Tuesday July 04, 2023 @07:00AM (#36584991)
Here are some notes I recently found online on this by simonw: https://simonwillison.net/2022/Oct/23/datasette-gunicorn/#be....

It seems to me like they removed the warning on SQLite, so I guess they tested it and it works fine. But I didn't research this any further so far.