Re:Ask HN: What’s the Next Big Thing?
by gavinray on Monday May 23, 2022 @01:02AM
(#31475478)
Language-specific drivers for datasources (like JDBC) being replaced with Arrow FlightSQL (gRPC by default, but transport agnostic)
Substrait as a standard for communicating query operations between systems
Data of the future will be exposing your datasource over FlightSQL endpoint so every language can use it, and consuming/producing Substrait IR for query plans
Substrait as a standard for communicating query operations between systems
Data of the future will be exposing your datasource over FlightSQL endpoint so every language can use it, and consuming/producing Substrait IR for query plans
https://arrow.apache.org/blog/2022/02/16/introducing-arrow-f...
https://substrait.io/
Apache Arrow and Facebook's experimental query engine Velox are already doing this