News for hackers. Stuff that matters.

A comment on DuckDB is probably the most important geospatial software of the last decade. Parent comment

Re:DuckDB is probably the most important geospatial software... by aynyc on Sunday May 04, 2025 @11:02AM (#43887103)
No, the EC2 instance doesn't have 500GB of data. Does DuckDB require that? I actually downloaded the data from S3 to local EBS and still choked.
[ Parent | on HN ]
Re:DuckDB is probably the most important geospatial software... by broner on Sunday May 04, 2025 @05:47PM (#43889861)
Works fine for me on TB+ datasets. Maybe you were doing in-memory rather than persistent database and running out of RAM? https://duckdb.org/docs/stable/clients/cli/overview.html#in-...
[ Parent | on HN ]
Re:DuckDB is probably the most important geospatial software... by aynyc on Sunday May 04, 2025 @09:52PM (#43891287)
Wait, do you insert the data from S3 into duckdb? I was just doing select from file.
[ Parent | on HN ]
Re:DuckDB is probably the most important geospatial software... by fastasucan on Monday May 05, 2025 @02:02PM (#43897726)
Maybe its your terminal that chockes because it tries to display to much data? 500GB should be no problem.
[ Parent | on HN ]
Re:DuckDB is probably the most important geospatial software... by broner on Monday May 05, 2025 @08:22PM (#43900723)
Nope, just reading from S3. Check this out: https://duckdb.org/2024/07/09/memory-management.html
[ Parent | on HN ]