Re:BigQuery prices increasing by more than 100% for many users
by rekttrader on Friday March 31, 2023 @11:18AM
(#35388382)
Export your data via avro files and try using Athena
[ Parent
| on HN ]
Re:BigQuery prices increasing by more than 100% for many users
by erikcw on Friday March 31, 2023 @11:24AM
(#35388511)
Ideally you’d used Parquet or ORC if querying from Athena/Trino/Presto. Since they are columnar formats you will enjoy considerably faster queues and lower query costs for most query patterns other than “select *…” since the query engine can just retrieve the columns needed for your query instead of the entire row.
Re:BigQuery prices increasing by more than 100% for many users
by mateo411 on Friday March 31, 2023 @06:22PM
(#35394537)
100 percent correct.
Ideally you will also partition the data, so your queries can use partitions. Athena charges $5 per terabyte of data scanned, so it's important to get this right.