News for hackers. Stuff that matters.
Re:Why Is SQLite Coded in C and not Rust by btilly on Tuesday October 14, 2025 @08:39PM (#45586707)
One of the things that SQLite is explicitly designed to do is have predictable behavior in a lot of conditions that shouldn't happen. One of those predictable behavior is that it does its best to stay up and running, and continuing to do the best it can. Conditions where it should succeed in doing this include OOM, the possibility of corrupted data files, and (if possible) misbehaving CPUs.

Automatic array bounds checks can get hit by corrupted data. Thereby leading to a crash of exactly the kind that SQLite tries to avoid. With complete branch testing, they can guarantee that the test suite includes every kind of corruption that might hit an array bounds check, and guarantee that none of them panic. But if the compiler is inserting branches that are supposed to be inaccessible, you can't do complete branch testing. So now how do you know that you have tested every code branch that might be reached from corrupted data?

Furthermore those unused branches are there as footguns which are reachable with a cosmic ray bit flip, or a dodgy CPU. Which again undermines the principle of keeping running if at all possible.

Re:Why Is SQLite Coded in C and not Rust by vlovich123 on Tuesday October 14, 2025 @10:44PM (#45587548)
In rust at least you are free to access an array via .get which returns an option and avoids the “compiler inserted branch” (which isn’t compiler inserted by the way - [] access just implicitly calls unwrap on .get and sometimes the compiler isn’t able to elide).

Also you rarely need to actually access by index - you could just access using functional methods on .iter() which avoids the bounds check problem in the first place.

Re:Why Is SQLite Coded in C and not Rust by OptionOfT on Wednesday October 15, 2025 @12:08AM (#45587975)
For slices the access is handled inside of the compiler: https://github.com/rust-lang/rust/blob/235a4c083eb2a2bfe8779...

I'm checking to see how array access is implemented, whether through deref to slice, or otherwise.

Re:Why Is SQLite Coded in C and not Rust by vlovich123 on Wednesday October 15, 2025 @12:10AM (#45587985)
I had Vec in mind but regardless nothing forces you to use the bounds-checked variant vs one that returns option<t>. And if you really are sure the bounds hold you can always use the assume crate or just unwrap_unchecked explicitly.
Re:Why Is SQLite Coded in C and not Rust by jemmyw on Tuesday October 14, 2025 @10:30PM (#45587469)
Keeping running if possible doesn't sound like the best strategy for stability. If data was corrupted in memory in a was that would cause a bounds check to fail then carrying on is likely to corrupt more data. Panic, dump a log, let a supervisor program deal with the next step, or a human, but don't keep going potentially persisting corrupted data.
Re:Why Is SQLite Coded in C and not Rust by hoppp on Tuesday October 14, 2025 @10:47PM (#45587570)
It still needs to detect that there is corrupted data, dump the log and the supervisor would not be the best if it was external since in some runtimes it could be missing, they just build it into it and we came full circle.
Re:Why Is SQLite Coded in C and not Rust by btilly on Wednesday October 15, 2025 @01:10AM (#45588288)
What the best strategy is depends on your use case.

The use case that SQLite has chosen to optimize for is critical embedded software. As described in https://www.sqlite.org/qmplan.html, the standard that they base their efforts on is a certification for use in aircraft. If mission critical software on a plane is allowed to crash, this can render the controls inoperable. Which is likely to lead to a very literal crash some time later.

The result is software that has been optimized to do the right thing if at all possible, and to degrade gracefully if that is not possible.

Note that the open source version of SQLite is not certified for use in aviation. But there are versions out there that have been certified. (The difference is a ton of extra documentation.) And in fact SQLite is in use by Airbus. Though the details of what exactly for are not, as far as I know, public.

If this documented behavior is not what you want for your use case, then you should consider using another database. Though, honestly, no other database comes remotely close when it comes to software quality. And therefore I doubt that "degrade as documented rather than crash" is a good reason to avoid SQLite. (There are lots of other potential reasons for choosing another database.)

Re:Why Is SQLite Coded in C and not Rust by Groxx on Wednesday October 15, 2025 @02:34AM (#45588777)
outside political definitions, I'm not sure "crash and restart with a supervisor" and "don't crash" are meaningfully different? they're both error-handling tactics, likely perfectly translatable to each other, and Erlang stands as an existence proof that crashing is a reasonable strategy in extremely reliable software.

I fully recognize that political definitions drive purchases, so it's meaningful to a project either way. but that doesn't make it a valid technical argument.

Re:Why Is SQLite Coded in C and not Rust by Izkata on Wednesday October 15, 2025 @10:42AM (#45593393)
If the cause of the crash is in any way related to the persisted data, there's a good chance you're now stuck in a crashloop.

If it can avoid crashing, other functionality may continue to work fine.

Re:Why Is SQLite Coded in C and not Rust by Groxx on Thursday October 16, 2025 @11:14AM (#45606426)
this is true for any kind of error branch - if the cause is persisted, it's going to still be there on the next iteration.
Re:Why Is SQLite Coded in C and not Rust by btilly on Wednesday October 15, 2025 @11:03AM (#45593720)
Yes, Erlang demonstrates that "crash and restart with a supervisor" is a potentially viable strategy to reliability.

But the choice is not just political. There are very meaningful technical differences for code that potentially winds up embedded in other software, and could be inside of literal embedded software.

The first is memory. It takes memory to run whatever is responsible for detecting the crash, relaunching, and starting up a supervisor. This memory is not free. Which is one of the reasons why Erlang requires at a minimum 10 MB or so of memory. By contrast the overhead of SQLite is something like half a MB. This difference is very significant for people putting software into medical devices, automotive controllers, and so on. All of which are places where SQLite is found, but Erlang isn't.

The second is concurrency. Erlang's concurrency model leaks - you can't embed it in software without having to find a way to fit Erlang concurrency in. This isn't a problem if Erlang already is in your software stack. But that's an architectural constraint that would be a problem in many of the contexts that SQLite is actually used in.

Remember, SQLite is not optimized for your use case. It is optimized for embedded software that needs to try to keep running when things go wrong. It just happens to be so good that it is useful for you.

Re:Why Is SQLite Coded in C and not Rust by jemmyw on Thursday October 16, 2025 @06:58AM (#45603886)
You're right and when I thought about it more I considered that "supervisor" isn't what I would want. Rather I'm thinking raising errors to the program that embeds sqlite so that it can decide what to do. I do have a desktop app that uses sqlite and I'd rather it raised an error than tried to recover.
Re:Why Is SQLite Coded in C and not Rust by ulfgard on Saturday October 18, 2025 @07:29AM (#45626496)
But getting back into a well-defined state which you can communicate via an error is a viable recovery strategy.

This state can be "the database is corrupt", allowing for a restore from backup, it can also be "shut down gracefully" and communicate that.

Especially for databases there is a ton to concinder: what about database entries that are not committed yet? When you run into oom, should you at least try to commit the data still in ram by freeing as much space as possible?