News for hackers. Stuff that matters.
Re:BigQuery prices increasing by more than 100% for many users by rekttrader on Friday March 31, 2023 @11:18AM (#35388382)
Export your data via avro files and try using Athena
Re:BigQuery prices increasing by more than 100% for many users by erikcw on Friday March 31, 2023 @11:24AM (#35388511)
Ideally you’d used Parquet or ORC if querying from Athena/Trino/Presto. Since they are columnar formats you will enjoy considerably faster queues and lower query costs for most query patterns other than “select *…” since the query engine can just retrieve the columns needed for your query instead of the entire row.
Re:BigQuery prices increasing by more than 100% for many users by mateo411 on Friday March 31, 2023 @06:22PM (#35394537)
100 percent correct.

Ideally you will also partition the data, so your queries can use partitions. Athena charges $5 per terabyte of data scanned, so it's important to get this right.