News for hackers. Stuff that matters.

A comment on Show HN: Query SQLite files stored in S3.

Re:Show HN: Query SQLite files stored in S3 by civilized on Tuesday September 13, 2022 @09:05PM (#32832075)
This is nice, but before you go to the trouble of using it, consider just putting your CSV, JSON, or parquet data file in S3 and query it with S3 Select https://docs.aws.amazon.com/AmazonS3/latest/userguide/select...

I haven't tried this myself but parquet + S3 select should be especially good for analytical queries.

[ Parent | on HN ]
Re:Show HN: Query SQLite files stored in S3 by grangerg on Tuesday September 13, 2022 @09:42PM (#32832328)
Seems like it should be. That's pretty much the status quo with how Azure Synapse works. You put your Parquet file out on storage, and query it (via Spark or Pandas).
[ Parent | on HN ]