News for hackers. Stuff that matters.
Re:Learn Postgres at the Playground – Postgres compiled to W... by samwillis on Wednesday August 17, 2022 @02:27PM (#32499930)
Amusingly you where replying to me there, hello again!

I’m sure there are rough edges right now, but I’m complete convinced that even if the api isn’t there yet, this use case will win out and we will see it happen.

From memory the teams working on WASM SQLite are working with the File System API working group to ensure their use case is supported.

Re:Learn Postgres at the Playground – Postgres compiled to W... by chrismorgan on Wednesday August 17, 2022 @02:45PM (#32500137)
[I retract this comment entirely, apart from the pleasantries and the preference of being wrong—see downthread where rewq4321 corrects me.]

Heh, didn’t notice the username match there!

I don’t think “rough edges” is the right characterisation. What OPFS provides is just nothing in the direction required. The kind of file system you need to build a database on is a fundamentally completely different beast, with only unimportant surface-level similarities. It would generally require a complete replacement of the backend, with quite possibly literally no code in common.

I’d like something like this, because it’s certainly genuinely useful for cases like this, but I’d honestly be surprised if it ever happens, because it’s just… not webby. To be useful, it just about requires that the whole thing be backed by an actual file system and exposing that, which is something that has been assiduously avoided so far in the design, probably in significant part because it discloses quite a lot about the host system (fingerprinting; disk performance characteristics, quite possibly even file system identification by various nuances in behaviour; and surprisingly large side-channel attack possibilities, mildly similar to the fuss over high-resolution timers), but also because it tends to be a security hazard, just another moving part where things can go wrong more easily than you imagine.

I strongly suspect it will end up a bit like Web SQL: a nice idea that pretty much everyone agrees is a nice idea, but which is also a non-starter for other reasons.

But I wouldn’t mind being wrong. I do want to be able to deploy a robust, high-performing SQLite in the browser.

Re:Learn Postgres at the Playground – Postgres compiled to W... by rewq4321 on Wednesday August 17, 2022 @03:17PM (#32500524)
Are you sure you're up to date on the latest additions to this API? This wasn't in the MVP:

> The origin private file system provides optional access to a special kind of file that is highly optimized for performance, for example, by offering in-place and exclusive write access to a file's content.

https://web.dev/file-system-access/#accessing-files-optimize...

It was originally going to be a separate high-perf "Storage Foundation" API, but that was merged with the File System Access API.

Re:Learn Postgres at the Playground – Postgres compiled to W... by chrismorgan on Wednesday August 17, 2022 @03:29PM (#32500678)
Huh. Disregard what I wrote entirely, then. Reading through https://github.com/WICG/file-system-access/blob/main/AccessH..., I can see how they’ve bypassed most or all of the problems I saw—I was making unnecessary assumptions.

Thank you for correcting me. I am now enthusiastic about OPFS.

Re:Learn Postgres at the Playground – Postgres compiled to W... by ankoh on Wednesday August 17, 2022 @05:25PM (#32501921)
This has to be ruled out first: https://github.com/whatwg/fs/issues/7#issuecomment-116176851...

...but then the OPFS will be a quite decent fit. We (DuckDB-Wasm) are also looking closely at OPFS.

IMHO the requirement here is not even to get to full ACID.

With OPFS, we will get close enough to IndexedDB on steroids and bypassing the js heap limits through out-of-core operators.

After all, we are still running in a browser.

So I see the value of Wasm-based databases to be a front-facing accelerator, not a substitute for robust storage solutions.

Re:Learn Postgres at the Playground – Postgres compiled to W... by SomaticPirate on Wednesday August 17, 2022 @09:46PM (#32504224)
Great to see this kind of pleasant discourse on HN!