News for hackers. Stuff that matters.

A comment on AthenaX – Uber Engineering’s Open Source Streaming Analytics Platform.

Re:AthenaX – Uber Engineering’s Open Source Streaming Analyt... by DandyDev on Tuesday October 10, 2017 @03:32AM (#15440004)
This looks like an amazing piece of software! I have a couple of questions:

- Does it (out of the box) support mapping JSON messages on a Kafka Topic to a table accessible from SQL?

- How would you support the following use case: run SQL queries on one or more Kafka Topics and expose the results in (near) real-time through a JDBC supporting BI tool? I assume you'd use, AthenaX to run the queries and store the results in a standard RDBMS like PostgreSQL?

[ Parent | on HN ]
Re:AthenaX – Uber Engineering’s Open Source Streaming Analyt... by haohui on Tuesday October 10, 2017 @03:49AM (#15440070)
(1) Yes. AthenaX provides a thin wrapper of KafkaJsonTableSource[1] out-of-the-box. As long as you specify the schema it should be good to go. (2) Your guess is correct. Internally we have AthenaX streaming data to MemSQL, Pinot or MySQL to support the use case you described. Some of them go through the JDBCTableSink [2], the others go through a customized connectors. AthenaX is designed to allow plugging in connectors when required.

[1] https://github.com/apache/flink/blob/master/flink-connectors... [2] https://github.com/apache/flink/blob/master/flink-connectors...

[ Parent | on HN ]