News for hackers. Stuff that matters.
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by c17r on Saturday September 19, 2020 @10:42PM (#24531597)
Man, I’ve wanted the “compatible SQL engine with only RAM storage” for testing for YEARS. Closest I got was some shenanigans with MSSQL’s LocalDB.
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by mStreamTeam on Saturday September 19, 2020 @10:50PM (#24531618)
I use docker to spin up new postgresql DBs on the fly
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by GordonS on Sunday September 20, 2020 @05:53AM (#24532980)
Same here, and I run the same set of migrations that run in production. To he clear, this is only done once per test session, not for individual tests, and the tests are written in such a way that they don't interfere with each other.

The overhead is actually pretty small, less than 10s. I'd saw too much for unit tests, but we'll within the tolerable range for integration/functional tests. Compared with the time I'd spend hacking together some brittle and unrealistic in-memory alternative, I much prefer to use a real database.

Re:DuckDB – An embeddable SQL database like SQLite, but supp... by toyg on Sunday September 20, 2020 @05:48AM (#24532951)
I do that too but it’s a noticeable overhead, I guess it wouldn’t be great for mass-testing scenarios.
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by striking on Saturday September 19, 2020 @10:55PM (#24531634)
RAM FS + fsync off?
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by viraptor on Saturday September 19, 2020 @11:22PM (#24531682)
Those solutions still have a high overhead. There's acid compliance, serialisation in memory, maintaining indices, and many other layers. Compare it to an ideal testing solution with no initialisation cost and insert being literally: parse the query, add a new entry to a list, done.
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by RantyDave on Saturday September 19, 2020 @11:39PM (#24531725)
But you want to be testing against something that is as close as possible to the deployment environment. So if that means acid, indices etc, then that's what it is.
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by viraptor on Sunday September 20, 2020 @12:01AM (#24531796)
You can still do them in a trivial way that works like production. For example: if some column has a unique index, look at all rows and compare the new value. You don't need an actual index for it. (And definitely not a fancy concurrent access btree) For transactions/mvcc you can literally make a copy of everything.
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by Avamander on Sunday September 20, 2020 @09:21AM (#24533860)
Have you tried postgresql with libeatmydata?
Re:DuckDB – An embeddable SQL database like SQLite, but supp... by xyzzy_plugh on Saturday September 19, 2020 @11:46PM (#24531744)
I'd build this, but I'm not sure if I'd be able to get anyone to fund it.