Ask HN: Have you used SQLite as a primary database?

Re:Ask HN: Have you used SQLite as a primary database? by bsenftner on Monday April 25, 2022 @07:19AM (#31152978)

I no longer work there, but an enterprise facial recognition system used by NGOs, and 3-lettered government agencies has SQLite as the sole datastore. I wrote a portion of the SQLite runtime logic, a simply key/value store used all over the software.

SQLite proved to be phenomenal. We spec'ed hardware with enough RAM to hold the FR DB in memory, and damn SQLite is fast enough to keep up with the optimized FR system performing 24M face compares per second. With a 700M face training set, SQLite also proved instrumental in reducing the training time significantly. These daze, if given the opportunity to choose a DB I always choose SQLite. I use SQLite for my personal projects, and I go out of my way to not use MySQL because SQLite is so much faster.

[ Parent | on HN ]

Re:Ask HN: Have you used SQLite as a primary database? by barryhennessy on Monday April 25, 2022 @07:50AM (#31153213)

Impressive numbers, thanks for sharing.

Out of interest, were you running on bare metal/cloud? And what kind of CPU was behind those 24M face compares per second?

[ Parent | on HN ]

Re:Ask HN: Have you used SQLite as a primary database? by bsenftner on Monday April 25, 2022 @08:13AM (#31153385)

Running on bare metal, and those numbers come from a 3.4 GHz i9. The system is a fully integrated single executable, with embedded SQLite. Since I left the firm a year ago, new optimizations have the facial compares down to 40 nanoseconds per face.

[ Parent | on HN ]