15k inserts/s with Rust and SQLite (2021)

Re:15k inserts/s with Rust and SQLite (2021) by VWWHFSfQ on Sunday April 02, 2023 @05:24PM (#35415383)

The schema:

    CREATE TABLE IF NOT EXISTS users (
        id BLOB PRIMARY KEY NOT NULL,
        created_at TEXT NOT NULL,
        username TEXT NOT NULL
    );

The data:

    let user = User {
        id: uuid::Uuid::new_v4(),
        created_at: chrono::Utc::now(),
        username: String::from("Hello"),
    };

If you can't insert 15,000 of these records per second then there's something wrong with your database. I'm aware that people are all on the SQLite hype train, but this kind of stuff is table-stakes.

I really like SQLite. But this blog isn't any kind of great performance indicator for it.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by snacktaster on Sunday April 02, 2023 @05:37PM (#35415517)

This entire post reduces to:

    insert into users (id, created_at, username) values ("<id>", "<created_at>", "Hello"), ...15,000 times.

And we're going to be impressed with sqlite's performance?

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by Aeolun on Sunday April 02, 2023 @08:37PM (#35417064)

Well, some people think 3 inserts per second is fast, so maybe it gives some perspective? To be fair, I wouldn’t expect those to hang around hackernews though.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by ZephyrBlu on Sunday April 02, 2023 @06:20PM (#35415890)

I think a lot of people have forgotten or don't know how powerful CPUs are. Simple operations should be insanely fast, like easily sub-millisecond if not sub-nanosecond fast.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by Aeolun on Sunday April 02, 2023 @08:36PM (#35417054)

You can double the performance of this whole thing by inserting 2 values at a time.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by eyegor on Sunday April 02, 2023 @09:56PM (#35417620)

I don't see why you wouldn't just bundle them all into a transaction at that point. The idea of 1 insert at a time is to roughly estimate the overhead/perf in a real environment, where you're only adding 1 entry at a time.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by ijidak on Sunday April 02, 2023 @09:11PM (#35417321)

But, for insert, the CPU is not the bottleneck. It's I/O constrained.

Disk I/O has always been the bottleneck for writes.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by jandrewrogers on Sunday April 02, 2023 @09:37PM (#35417515)

Counterintuitively, write performance in modern databases is primarily constrained by disk read performance.

That said, it is increasingly rare for disk I/O to be a bottleneck for inserts. When your storage will happily sustain 10+ GB/s of write throughput, you tend to run out of things like network and memory bandwidth first.

[ Parent | on HN ]

Re:15k inserts/s with Rust and SQLite (2021) by zhte415 on Monday April 03, 2023 @12:53AM (#35418692)

I think it's useful stress testing for when your product goes viral.

For example, if the whole world wanted to create an account, and an account consisted of a UUID and a date, it'd take 5.4 days using this metric as a minimum viable benchmark.

While this is a tongue in cheek comment, it has a valid foundation: 15,000 seems like a large number, and 1 second seems like a small amount of time. On the other hand, 5.4 days seems like quite a large number. Likewise at small scales, 15,000/second operated at a fidelity only able to capture 0.0005% of a typical processor's clock.

It's all about context.

This is not to say that the ability to casually write 15,000 things a second to a database is not impressive in itself.

[ Parent | on HN ]