News for hackers. Stuff that matters.
Re:SQLite is all you need for durable workflows by tasuki on Friday May 29, 2026 @04:59PM (#48329186)
> SQLite can work well for the parts of your system where there is naturally strong partitioning.

Or the parts of your system that don't have big data and no need for massively concurrent writes. And that's the vast majority of systems!

Re:SQLite is all you need for durable workflows by MattJ100 on Friday May 29, 2026 @06:06PM (#48329938)
You can do big data in SQLite. Concurrent writes, sure, I'd recommend something else.

If you think the majority of systems require massively concurrent writes, I think you need to look a bit harder. SQLite is, after all, the most widely deployed database system, ever.

Re:SQLite is all you need for durable workflows by therealdrag0 on Friday May 29, 2026 @06:33PM (#48330212)
It’s widely deployed as a local DB for local apps like phones, desktops, and web browsers. But it’s not the most used for distributed, concurrent web apps, which db servers were designed for. Maybe people are talking past each other, but that’s the debate I see.
Re:SQLite is all you need for durable workflows by kellogah on Friday May 29, 2026 @07:31PM (#48330733)
Not sure why there’s a debate at all. The discussion is on using SQLite instead of jq and markdown files. People got lost on a tangent! :)
Re:SQLite is all you need for durable workflows by gnabgib on Friday May 29, 2026 @07:51PM (#48330857)
No it's not.. the context of other threads on this post (which mention jq) do not apply here. How poorly coded are you?
Re:SQLite is all you need for durable workflows by kellogah on Friday May 29, 2026 @08:19PM (#48331031)
Like how poorly did Jesus code my DNA? Ask him. By him I mean ChatGPT Jesus mode.
Re:SQLite is all you need for durable workflows by petre on Saturday May 30, 2026 @12:40AM (#48332649)
We recently just partitioned the data into many SQLite databases and got away with it. It's telemetry data from IoT devices: one device, one database. Backups are an easy rsync job now instead of streaming a multi gigabyte database with compression that take hours. Reporting will just open each database and aggregate multi device data into another database (Duckdb, SQLite or something else, we'll see). Duckdb is not readable when locked so it's probably also going to be SQLite. Even it it's going to spit out JSON it will go into SQLite rows instead of many files.
Re:SQLite is all you need for durable workflows by datadrivenangel on Saturday May 30, 2026 @01:14AM (#48332827)
Check out Quack for DuckDB.
Re:SQLite is all you need for durable workflows by petre on Monday June 01, 2026 @06:31AM (#48354951)
I saw that but it's a bit too much friction. We'd rather attach a SQLite database and copy the results into that. Also no worries about version changes and compatibility, the types are not too restrictive. But we'll see.
Re:SQLite is all you need for durable workflows by sharts on Saturday May 30, 2026 @02:46AM (#48333354)
Internet Explorer 6 was the most widely deployed awesome piece of software. Those that hated it need to look a bit harder.
Re:SQLite is all you need for durable workflows by Gud on Saturday May 30, 2026 @03:54AM (#48333787)
It was not really “deployed” by a lot of people in the same sense.

It was forced upon most of us(not me, I used BeOS then Debian then FreeBSD).

I deployed phoenix.

Re:SQLite is all you need for durable workflows by HighGoldstein on Saturday May 30, 2026 @08:13AM (#48335362)
The reason SQLite is the most deployed is that it's used by Android.
Re:SQLite is all you need for durable workflows by velcrovan on Saturday May 30, 2026 @11:46AM (#48337478)
…and iOS, and Windows, and Mac OS, and Boeing, and Sony, and Firefox and Chrome and Safari…
Re:SQLite is all you need for durable workflows by HighGoldstein on Saturday May 30, 2026 @11:54AM (#48337558)
Yes, which goes in line with the argument that claiming that it's "the most deployed" as proof of superiority or suitability for any use case is equivalent to claiming the same for Internet Explorer. It's the most deployed because it's bundled in a lot of systems, not because people are purposefully using it as a DBMS.
Re:SQLite is all you need for durable workflows by velcrovan on Saturday May 30, 2026 @04:11PM (#48340158)
But it doesn't, because none of those systems are presenting SQLite to the user as something they should be using; they don't even make SQLite available to the user at all. Those systems all use SQLite internally to manage data.
Re:SQLite is all you need for durable workflows by andersmurphy on Saturday May 30, 2026 @02:43PM (#48339421)
What's massive amount concurrent writes? What's big data?
Re:SQLite is all you need for durable workflows by whatisthiseven on Sunday May 31, 2026 @07:53AM (#48345011)
Such an exercise is left for the reader.