Re:Faster PostgresSQL to BigQuery
by pantsforbirds on Tuesday January 10, 2023 @12:37PM
(#34328433)
I really love working with parquet and the general arrow ecosystem. The performance and cost ratios you can get out of it are really insane. AWS S3 + parquet + athena is one of the best and cheapest databases I've ever used.
[ Parent
| on HN ]
[I work on BigQuery]
BigQuery also supports in-place querying of datasets on GCS (or S3/Azure using Omni) via external/BigLake tables. https://cloud.google.com/bigquery/docs/query-cloud-storage-u...
Re:Faster PostgresSQL to BigQuery
by nlittlepoole on Friday January 13, 2023 @04:48PM
(#34374360)
Can we get any of Iceberg/Delta/Hudi that isn't terribly complex to setup? Like configurable completely from Standard SQL via the CREATE EXTERNAL TABLE syntax.