News for hackers. Stuff that matters.
Re:SQLite the only database you will ever need in most cases by crazypython on Thursday April 15, 2021 @11:44AM (#26821875)
Firefox uses SQLite for localStorage. When Firefox is force quit, the database becomes corrupted. https://bugzilla.mozilla.org/show_bug.cgi?id=1341070

This bug has occurred to me personally multiple times.

Does it really have advantages over a flat file?

Re:SQLite the only database you will ever need in most cases by NicoJuicy on Thursday April 15, 2021 @01:13PM (#26823315)
Sqlite is certified for use in planes. This requires a high level of tests.

I think the odds are likely that Firefox is doing something unsuspected.

https://www.sqlite.org/testing.html

- Four independently developed test harnesses

- 100% branch test coverage in an as-deployed configuration

- Millions and millions of test cases

- Out-of-memory tests

- I/O error tests

- Crash and power loss tests

- Fuzz tests

- Boundary value tests

- Disabled optimization tests

- Regression tests

- Malformed database tests

- Extensive use of assert() and run-time checks

- Valgrind analysis

- Undefined behavior checks

- Checklists

Re:SQLite the only database you will ever need in most cases by easton on Thursday April 15, 2021 @03:07PM (#26824854)
"Valgrind is perhaps the most amazing and useful developer tool in the world."

It seems that about once a month I read something that makes me like SQLite even more. I guess that they like Valgrind as much as I do is the reason this month.

Re:SQLite the only database you will ever need in most cases by unmole on Thursday April 15, 2021 @11:25PM (#26829067)
Valgrind is fine if you can live with the false positives, limited stack validation and execution slowdown. In a development environment, AddressSanitizer is a far better choice.
Re:SQLite the only database you will ever need in most cases by throwaway2037 on Thursday April 15, 2021 @03:19PM (#26824967)
I am trolling on the person you to whom you replied...

<<This bug has occurred to me personally multiple times.>>

Hello, I am AirBus engineer (not really, only pretend). Does my plane crash? No, it does not!

Dear Random Internet Person who complains that open source software is broken. Are you joking? We are talking about SQLite!!??

SQLite has whole pages dedicated to "why you should NOT use SQLite". How many open source projects are /so/ good they can do this? Incredible!

The test coverage feels like NASA.

Re:SQLite the only database you will ever need in most cases by throwawayboise on Thursday April 15, 2021 @12:26PM (#26822584)
I use Firefox almost exclusively and have never encountered this.

Whatever capabilities SQLLite has to recover corrupted or inconsistent storage, it certainly exceeds what you get with flat files (none, unless you implement it yourself).

Re:SQLite the only database you will ever need in most cases by btilly on Thursday April 15, 2021 @12:39PM (#26822793)
Per that bug report, you are likely to encounter it at https://codesandbox.io/s/sentry-js-sdk-unhandled-promise-fuq....

Which suggests an application error to me. I'd give decent odds to the real problem being some bounds check or pointer writing to the wrong place. To prove it, I'd instrument SQLite to log every single query that it was asked to do, then try to run those queries outside of the application. If the queries work fine, then the copy of SQLite in the application is being corrupted somehow.

In which case it really isn't fair to blame SQLite.

Re:SQLite the only database you will ever need in most cases by mcdevilkiller on Friday April 16, 2021 @11:49AM (#26835390)
What's getting corrupted is Firefox's own SQLite DB.
Re:SQLite the only database you will ever need in most cases by btilly on Friday April 16, 2021 @09:23PM (#26840837)
I'm perfectly aware of that.

The question is how it is getting corrupted. Is it that SQLite has a bug? Or the copy of SQLite in Firefox is getting corrupted first?

Given the complexity and relative levels of testing of those two products, my bet is that the problem is Firefox.

Re:SQLite the only database you will ever need in most cases by astrobe_ on Thursday April 15, 2021 @12:22PM (#26822493)
This is quite surprising. SQLite is also used in embedded where you don't have comparatively civil "forced quits" but also straight power offs. SQLite seems to do everything it can to handle this gracefully.
Re:SQLite the only database you will ever need in most cases by edoceo on Thursday April 15, 2021 @11:56AM (#26822062)
if you're corrupting sqlite files you're for sure going to corrupt other formats in a force quit situation.
Re:SQLite the only database you will ever need in most cases by ravenstine on Thursday April 15, 2021 @03:09PM (#26824868)
Why would that not happen to any other form of flat file storage? When you force quit a process, unless there's some nuance I'm unaware of, you're not only bypassing any graceful cleanup but there could be an FS operation in progress. Although I'm certain there are mitigation techniques, I would never expect a process to guarantee no file corruption when it's forcefully quit.

Interestingly enough, I don't think I've ever had this issue. I've force quit Firefox many times in the past (not often, but I'm sure I've done it countless times in the 10+ years I've been using it) and I still have localStorage data intact. There's old crap in StackEdit.io, which uses localStorage, that I haven't touched in years. (no, I have neither that service or Firefox itself set up to sync with anything)

EDIT: Maybe it's OS specific? I've been on macOS, and maybe there's something about other file systems where corruption is more likely to happen with a force-quit? Just throwing darts here.

Re:SQLite the only database you will ever need in most cases by asimpletune on Thursday April 15, 2021 @03:30PM (#26825104)
I believe for a proper implementation it would be done transactionally. So if the program crashed then that record wouldn’t be written but the DB wouldn’t be corrupted.
Re:SQLite the only database you will ever need in most cases by nix0n on Thursday April 15, 2021 @03:30PM (#26825101)
I force-quit Firefox regularly (easiest way to get my tabs to stay open), on both Linux and Windows.

I've never seen any error messages from Firefox suggesting that anything has been corrupted.

Are there other symptoms that I would expect to see, if this corruption is happening?

Re:SQLite the only database you will ever need in most cases by crazypython on Friday April 16, 2021 @01:48PM (#26836987)
Some sites break due to not expecting the exception being thrown– deviantart.com won't work, for instance.