Postgres and Parquet in the data lake

Re:Postgres and Parquet in the data lake by bjackman on Thursday May 05, 2022 @05:14AM (#31271215)

For [Ex-]Googlers: I thought this sounded a bit like Capacitor and indeed it is mentioned in this page:

https://cloud.google.com/blog/products/bigquery/inside-capac...

(I don't think there's a direct analogue to Postgres though? Maybe people sometimes take data out of Spanner and put it in a Capacitor file instead sometimes?)

[ Parent | on HN ]

Re:Postgres and Parquet in the data lake by flakiness on Thursday May 05, 2022 @09:37AM (#31273141)

And for Non-Googlers: BigQuery does support Parquet in the lake. If you don't need PQ compatibility this is an easy option. https://cloud.google.com/bigquery/external-data-cloud-storag...

People dump data from the main OLTP to the "lake" or some OLAP database and call it data-warehouse vs. Internet-oriented companies like Google tend to focus more on querying the log and its aggregated data.

[ Parent | on HN ]