SQL pipe syntax available in public preview in BigQuery

Re:SQL pipe syntax available in public preview in BigQuery by jakozaur on Thursday February 13, 2025 @11:06AM (#43037330)

The SQL with pipe syntax is also been implemented at Databricks since Jan 30, 2025: https://docs.databricks.com/en/sql/language-manual/sql-ref-s...

Still, the best is yet to come. Previously, SQL extensions were a pain. There was no good place, and table-value functions were a mess.

Now, it would be possible to have higher-order functions such as enrichment, predictions, grouping or other data contracts. Example:

  FROM orders
  |> WHERE order_date >= '2024-01-01'
  |> AGGREGATE SUM(order_amount) AS total_spent GROUP BY customer_id
  |> WHERE total_spent > 1000
  |> INNER JOIN customers USING(customer_id)
  |> CALL ENRICH.APOLLO(EMAIL > customers.email)
  |> AGGREGATE COUNT(*) high_value_customer GROUP BY company.country

This may be called one SQL to determine distinct e-mail domains, then prepare an enriching dataset and later execute the final SQL with JOIN.

Iterative SQL with pipes may also work better with GenAI.

[ Parent | on HN ]

Re:SQL pipe syntax available in public preview in BigQuery by memhole on Thursday February 13, 2025 @03:33PM (#43040953)

Thanks for sharing. I wasn't aware they pushed that out. The ordering of this makes so much more sense. My only real concern, is that I think CTEs are so common place and part of ANSI SQL that you'll see people trade a standard for this. But, I also gripe that Snowflake uses // for comments and people get confused when their code doesn't work on another DB. Or Oracles' join syntax is another example.

[ Parent | on HN ]

Re:SQL pipe syntax available in public preview in BigQuery by mc_maurer on Thursday February 13, 2025 @05:56PM (#43042541)

If only it worked on Databricks serverless SQL warehouses...

[ Parent | on HN ]

Re:SQL pipe syntax available in public preview in BigQuery by rxin on Thursday February 13, 2025 @10:42PM (#43044595)

This is coming to Serverless SQL warehouses soon!

[ Parent | on HN ]

Re:SQL pipe syntax available in public preview in BigQuery by valyala on Thursday February 13, 2025 @07:07PM (#43043083)

Pipe-based query syntax has been appeared a better alternative to SQL in many databases, because it is easier to read, write and maintain. The only thing is that this syntax isn't standardized among databases yet.

BTW, we at VictoriaLogs also use pipe-based syntax for LogsQL query language [1] since the first release in 2023. Recently I wrote an SQL -> LogsQL conversion guide [2], which has been appeared quite clear and easy to follow. This guide was written during the conversion of SQL queries to LogsQL at ClickBench benchmark from ClickHouse [3].

[1] https://docs.victoriametrics.com/victorialogs/logsql/

[2] https://docs.victoriametrics.com/victorialogs/sql-to-logsql/

[3] https://github.com/ClickHouse/ClickBench/tree/main/victorial...

[ Parent | on HN ]