News for hackers. Stuff that matters.
Re:DataFusion Comet: Apache Spark Accelerator by andygrove on Friday May 31, 2024 @04:59PM (#40540166)
I wish I'd known about the search issue.

I do see a new opportunity for Ballista. By leveraging all of the Spark-compatible operators and expressions being built in Comet, it would be able to support a wider range of queries much more quickly.

Ballista already uses protobuf for sending plans to executors and Comet accepts protobuf plans (in a similar, but different format).

Re:DataFusion Comet: Apache Spark Accelerator by OutOfHere on Friday May 31, 2024 @06:26PM (#40540977)
Did Databricks sponsor Comet?
Re:DataFusion Comet: Apache Spark Accelerator by andygrove on Friday May 31, 2024 @10:52PM (#40542501)
No.