News for hackers. Stuff that matters.
Re:The SQL query engine Trino (formerly PrestoSQL) recaps a ... by dmead on Thursday August 04, 2022 @01:40PM (#32345924)
I do the support for my department's trino cluster. We move ~1tb (and growing) in ETL jobs and support interactive queries for the data scientists/analysts.

It would be super good if you guys added big query write support. Its really annoying to have to run a hive cluster in google to act as a proxy for this.

Re:The SQL query engine Trino (formerly PrestoSQL) recaps a ... by hashhar on Thursday August 04, 2022 @01:47PM (#32345995)
BigQuery very recently announced their Storage Write API which is one of the ways we were looking to implement this but there are some issues with the latency and consistency guarantees that it offers.

But, yes, we do plan to add that eventually after ironing out all the kinks. See https://github.com/trinodb/trino/pull/13094

Re:The SQL query engine Trino (formerly PrestoSQL) recaps a ... by bitsondatadev on Thursday August 04, 2022 @02:19PM (#32346427)
Also, you can keep track of all the BQ progress here: https://github.com/trinodb/trino/issues/6867
Re:The SQL query engine Trino (formerly PrestoSQL) recaps a ... by dmead on Friday August 05, 2022 @12:06PM (#32358066)
thanks. If we can eliminate the costs and upkeep of hive in gcp, it would make my life easier for sure.
Re:The SQL query engine Trino (formerly PrestoSQL) recaps a ... by atwebb on Thursday August 04, 2022 @01:41PM (#32345934)
Any chance you have an overview of the architecture and operations support required? How many data sources are you pinging?
Re:The SQL query engine Trino (formerly PrestoSQL) recaps a ... by dmead on Friday August 05, 2022 @12:05PM (#32358056)
like 20 different sources.

I nag my service reps about it. I"m sure it's been filed in your jira.