News for hackers. Stuff that matters.

A comment on Postgres WASM.

Re:Postgres WASM by mytherin on Monday October 03, 2022 @11:21AM (#33068766)
Looks really nice, but I'm really curious about what hurdles you encountered in making a native WASM version of Postgres. It seems that both the performance and binary size suffer immensely from the VM running underneath. For example, if I compare to DuckDB-WASM [1] there is an immense difference in load time and query execution speed.

Are there future plans at creating a native WASM version of Postgres? Making it run many times faster would certainly open up a lot more use cases.

(Disclaimer: I work on DuckDB, but have not worked on the WASM version myself)

[1] https://shell.duckdb.org

[ Parent | on HN ]
Re:Postgres WASM by pistoriusp on Monday October 03, 2022 @11:34AM (#33068986)
Snaplet person here: Having a native WASM version would be a huge win for everyone, especially is storage is decoupled from compute!

At the moment the CPU and memory snapshot of the VM (with Postgres) is 12 MB, and subsequent reloads are cached. So yeah, not the worst, but not great.

An optimization is that we're using 9P filesystem. So accessing anything on disk is lazily loaded over the network.

> Are there future plans at creating a native WASM version of Postgres?

Yup! I think that should be the goal, and we (Supabase & Snaplet) would be very happy to work with anyone that wants to build towards that.

[ Parent | on HN ]
Re:Postgres WASM by chrisjc on Monday October 03, 2022 @12:06PM (#33069423)
> especially is storage is decoupled from compute!

This would be amazing! I can imagine a situation where external tables are managed by some MPP, and a WASM compute engine (Postgres, DuckDB, etc) would be able to at least read subsets/partitions of the full external table.

I wonder if the work required to make a native WASM Postgres would have to be split up into efforts for row-based vs column-based. Selfishly, I would love to have access to a column-based version first.

[ Parent | on HN ]
Re:Postgres WASM by pistoriusp on Monday October 03, 2022 @01:16PM (#33070570)
We actually kinda see parts of this in https://github.com/superfly/litefs, albeit for SQLite.

And here we see some ideas forming around "pluggable storage for PostgresQL": https://wiki.postgresql.org/wiki/Future_of_storage#Pluggable...

Seriously! If any of this sounds interesting to build, reach out, and we'll make it happen!

[ Parent | on HN ]
Re:Postgres WASM by kiwicopple on Monday October 03, 2022 @11:28AM (#33068896)
congrats to the DuckDB team for their WASM version. I've been following it from afar and it's very impressive.

> what hurdles you encountered in making a native WASM

I'm sure Mark & Peter can jump in with specifics but mostly it was due to complexity - there it probably can be done it's just that we took the path of least resistance.

> Are there future plans at creating a native WASM version of Postgres

We'd like that. If anyone would like to collaborate with Supabase + Snaplet to create a more "native WASM" version then please reach out

[ Parent | on HN ]
Re:Postgres WASM by burggraf on Monday October 03, 2022 @11:33AM (#33068962)
Supabase developer here. Yes we'd love to collaborate on a native WASM version. I tried this first before starting on this project and I didn't get very far. I'll have to go back to my tests on that, but it seemed like WASM wasn't yet capable of compiling Postgres due do some libraries not being supported. Hopefully we can find workarounds or push WASM into a new era of compatibility :)
[ Parent | on HN ]