News for hackers. Stuff that matters.

A comment on Observable 2.0, a static site generator for data apps. Parent comment

Re:Observable 2.0, a static site generator for data apps by recifs on Friday February 16, 2024 @03:06AM (#39394427)
See the example at https://huggingface.co/spaces/observablehq/fpdn where DuckDB is used both as a data loader (to download and digest 200GB worth of source data into a small 8MB parquet file) and on the client-side to allow the user to do live search queries on the minimized data. Server-side, we're using duckdb-the-binary, and client-side we're using duckdb-wasm.
[ Parent | on HN ]
Re:Observable 2.0, a static site generator for data apps by kuatroka on Friday February 16, 2024 @05:05AM (#39395147)
So the 200Gb loading and digesting part is totally separate from the Observable Framework, right? You just do it with a standard ( non wasm duckdb as part of ETL) and later you just direct Observable Framework to read and plot the 8Gb file? Thanks
[ Parent | on HN ]
Re:Observable 2.0, a static site generator for data apps by severo_bo on Friday February 16, 2024 @07:37AM (#39396134)
nope, Observable Framework data loader accesses the 200GB dataset. The code is here: https://huggingface.co/spaces/observablehq/fpdn/blob/main/do...
[ Parent | on HN ]