News for hackers. Stuff that matters.
Re:I found a bug in SQLite by capableweb on Sunday December 11, 2022 @02:00PM (#33946231)
> larger than the codebase test suite

This tend to be true for most serious projects, that the amount of test code is greater than that of the code that is being exercised.

I think what they are famous for is the quality of the testing suite, rather than the amount.

Re:I found a bug in SQLite by loeg on Sunday December 11, 2022 @03:07PM (#33946925)
This is a definition of “serious projects” that excludes almost all serious projects.
Re:I found a bug in SQLite by rat9988 on Sunday December 11, 2022 @07:29PM (#33949304)
I'm not sure, all of mines at big companies have followed this definition.
Re:I found a bug in SQLite by valray on Sunday December 11, 2022 @03:08PM (#33946935)
> This tend to be true for most serious projects, that the amount of test code is greater than that of the code that is being exercised.

Reading this comment, I was thinking "Oh that must mean the test code is 2x or maybe even 3x the amount of source code"

Going to the SQLite web site, I was surprised to find that the test code is 600x larger than the source code. Impressive.

Is this 600:1 ratio typical for other projects? The ones that I have seen are more like 1x or 2x, but I have not worked with many open source systems.

Re:I found a bug in SQLite by baq on Sunday December 11, 2022 @03:43PM (#33947273)
They count post-codegen test LOC, the checked in ratio is much smaller - used to be something like 5:1. Still super impressive!
Re:I found a bug in SQLite by civopsec on Sunday December 11, 2022 @04:06PM (#33947508)
Test to regular code ratio shouldn’t be impressive by itself anyway, considering how repetitive, duplicated, and setup-heavy test code can be.
Re:I found a bug in SQLite by markerz on Sunday December 11, 2022 @02:09PM (#33946341)
I think a good way of looking at the quality of their tests is how they categorize their tests, which focuses on the different types of problems they want to solve: power loss, concurrency, API issues, logic, etc. https://www.sqlite.org/testing.html