Question: is it really persisted indefinitely? And will this SQLite file be portable (import and export to somewhere else) ?
https://webkit.org/blog/12257/the-file-system-access-api-wit...
https://web.dev/storage-for-the-web/#:~:text=Starting%20in%2....
> ...This eviction policy does not apply to installed PWAs that have been added to the home screen.
sqlite's storage format is independent of the underlying device. OPFS is "just another backend" and the only tricky part of implementing it was that OPFS's API is largely asynchronous and sqlite3 requires synchronous I/O. The db that's stored in OPFS is exactly what would be stored on your hard drive if you were operating outside of the browser.
"Mozilla and Microsoft TRIED to block it, but COULDN'T stop it!"
Seriously though, I can't wait to use this for my website. This is going to be ridiculously awesome combined with htmx.
(All of this is as far as I understand things, I could be completely wrong.)
1. You can persist data with localStorage/IndexDB, can't you? 2. There is nothing in the article about syncing. There is no out-of-the-box solution, as I can see.
I guess so, but given the situation that I'd already be using WASM SQLite in the client's browser, then I'd have to implement that part myself, or use something like Absurd SQL. I'd rather use the implementation made by the creator of SQLite.
> There is nothing in the article about syncing.
Correct, that's something I'll have to do myself. I don't see any issues there though, I'd just need to verify the user is authorised to make those changes, which in my specific use case seems pretty trivial.
Sorry, but it's just you opinion. Do you have proofs of how t-shirt store will benefit of putting RDBMS on a client side (how much space does it take BTW) instead of using simple list of objects?
And you still haven't answered the questions.
Edit: Additionally, I encourage you to experiment with this. If you haven’t already, you may be surprised at how efficient/compact SQLite files are. They’re significantly more compact that JSON or XML documents and can include binary files like images. It’s much simpler to use SQLite as a file format than to create a custom binary file format.
Think of something like personal finance, accounting, payroll, project management, CRM, zettelkasten style note taking, health and fitness tracking, etc, where users may want flexible analysis and reporting tools.
Much of this is now done on the backend, which makes sense if the data is accessed by many users in a transactional manner. But if it's a single user app or something used by tiny teams then cutting latency down to zero could greatly improve usability while still benefitting from everything a real query engine has to offer.
We’ve made one: https://electric-sql.com
Active-active SQLite to Postgres with transactional causal+ consistency based on CRDTs.
Disclaimer: founder.
I previously experimented with combining Yjs (CRDT toolkit) with Pouch/CouchDB to create an eventually consistent db, but decided that CouchDB was the wrong backend.
It looks like you have built exactly what I wanted to do if I had had time.
https://electric-sql.com/docs/overview/technical-intro
I happen across AntidoteDB just the other day, and my take-away was that it was still under development but not really ready for production use. Curious your opinion / experience with it?
It’s not production ready and neither are we (we’re in developer preview mode, which is like a public alpha [0]).
There are also other aspects on the Antidote roadmap such as efficiently materialising consistent secondary indexes that are ongoing challenges but aren’t so relevant to how we’re using it as a replication layer.
We are working, alongside others, with the Antidote developers to help fix these issues and generally improve reliability / engineer correct behaviour under load. (Professor Annette Bieniusa, who leads the development, is our Chief Architect). We have a fork at electric-sql/vaxine [1].
We are also taking advantage of some simplifications which mitigate the known issues. We have an #antidote channel in the ElectricSQL discord [2] if you’d be interested in chatting more.
[0]: https://electric-sql.com/docs/overview/faqs
You can also follow the generic driver integration instructions [0]. Basically need to implement an adapter interface and compose the various utilities.
Happy to help with this — shout on Discord [1] if you’d be interested in collaborating on it.
How heavy is the download? What's the minified client size at the moment?
Download size depends on the driver (we support different SQLite drivers for different environments). The web is quite heavy as it loads the SQL.js WASM (as a separate file, it’s not bundled). I want to say ~350-400kb total but I need to check and it depends a bit on how you build/bundle and serve it.
One approach would be to look at the network tab of the browser console when running the web example: https://github.com/electric-sql/examples
is that basically a desktop app that doesn't need to be installed? You still download it using a browser but then you don't install it on your operating system you just run it in your browser?
Another CRDT + SQLite project as a native extension: https://github.com/aphrodite-sh/cr-sqlite
Should have a release out here shortly but feel free to browse the code, tests and readme till then to see how it works.
i recently had to work with WebSQL in the context of comparing it to sqlite's new WASM support (of which i'm the developed). WebSQL, quite frankly, is a toy. Its execution model is far too limited and excludes all sorts of functionality, not the least of which is that it's impossible to delete a WebSQL db.
> yet here we are, back to that same spot, but with less performance and more complexity
Wrong. We have benchmarked the two in apples-to-apples comparisons, taking into account WebSQL's limitations. The two approaches are roughly equivalent, with both winning out under certain loads, despite WebSQL being implemented in native code.
Also even at similar performance you still need to download a bunch of extra stuff to even run the WASM version. My whole webpage takes less than it...
We is the sqlite team. i'm the "JS/WASM Guy" for the project.
> and where I can see those benchmarks ?
You can't currently because we don't have them in a user-consumable form. We've done a tremendous amount of benchmarking during the development because All The Speed was one of our design goals. However, all such records were in transient spreadsheets intended for one-shot note-taking use, not publication.
Once our documentation effort settles down, and responding to user feedback from the initial announcement slows down, i hope to implement a benchmarking application similar to:
<https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html>
Until then, however, you'll simply have to (A) take my word for it, (B) try it out yourself, or (C) none of the above, as you wish. Edit: or (D): we have a WASM port of sqlite's standard benchmarking tool, known as "speedtest1", in the sqlite source tree, but getting it up and running requires reading a good deal of documentation:
<https://sqlite.org/src/dir/ext/wasm&ci=trunk>
That tool is how we've benchmarked it so far, with the exception of comparing it to WebSQL, which required a custom application which is also in that directory (batch-runner.*). batch-runner, however, is in no way user friendly.
Is there a required feature that only OPFS provides, other than file locking (which seems like a pretty non-essential feature tbh)?
This is really good news, and exactly what the OPFS was designed for.
You may have seen “Absurd SQL” [1] which was a proof of concept for building a SQLite Virtual FS backend using IndexedDB. It provided full ACID compliment transactions. Incredible work but a hack at best.
The OPFS supersedes all that and makes it possible to have proper consistent and resilient transactions.
WASM SQLite with the OPFS is the future of offline first web app development. The concept of a single codebase web/mobile/desktop app with proper offline storage is here.
What I really want to see next is an eventually constant sync system between browser and server (or truly distributed with WebRTC). The SQLite Session Extension [2] potentially has the building blocks needed for such a system.
0: https://webkit.org/blog/12257/the-file-system-access-api-wit...
1: https://github.com/jlongster/absurd-sql
2: https://www.sqlite.org/sessionintro.html