AthenaX – Uber Engineering’s Open Source Streaming Analytics Platform

Re:AthenaX – Uber Engineering’s Open Source Streaming Analyt... by nevi-me on Monday October 09, 2017 @09:40PM (#15438700)

I share your sentiment here. Having a common base to work from also has the benefit that end-users, the BI or Data Science practitioner (whom we empower as data engineers) can expect a stable SQL dialect that they'll be able to use everywhere*.

I haven't followed Spark in recent months, but what I recall was that the SQL DSL had some caveats at first, because certain things weren't yet implemented.

My anecdote has been that projects that implement SQL after a while, typically don't deliver the whole thing on initial release. I imagine it's often quite a lot of work.

The more projects that use Calcite, the more upstream contributions there would be ...

[ Parent | on HN ]

Re:AthenaX – Uber Engineering’s Open Source Streaming Analyt... by rxin on Monday October 09, 2017 @11:01PM (#15438978)

Spark actually has probably the best standard SQL support among all open source big data frameworks. It can run all of TPC-DS queries without modifications.

[ Parent | on HN ]

Re:AthenaX – Uber Engineering’s Open Source Streaming Analyt... by haohui on Tuesday October 10, 2017 @03:39AM (#15440026)

TPC-DS evaluates against batch processing and it is great to hear that Spark supports all of them.

I think the space for streaming processing is still quickly evolving. Many features like stream-table joins, CTEs, streaming joins with late arrival data are unimplemented or do not even have clear semantics yet. It would be great to see a benchmarks like TPC-DS in the domain.

[ Parent | on HN ]