News for hackers. Stuff that matters.
Re:KSQL: Open Source Streaming SQL for Apache Kafka by stuartaxelowen on Monday August 28, 2017 @10:12PM (#15121638)
Doesn't PipelinDB limit you to postgres' scaling capabilities?
Re:KSQL: Open Source Streaming SQL for Apache Kafka by Fergi on Monday August 28, 2017 @10:28PM (#15121686)
Not for streaming analytic workloads, because PipelineDB fundamentally adds continuous queries to PostgreSQL, so data is continuously distilled and aggregated as it arrives, before it is stored, which drastically reduces the amount of data stored in PipelineDB (or soon, in PostgreSQL via the extension refactor).

PipelineDB also offers a clustering extension for large workloads (see: http://enterprise.pipelinedb.com/docs/)

But in terms of ad hoc, exploratory analytics workloads, yes - the scaling limitations would be the same, since for ad hoc, exploratory analytics PipelineDB and PostgreSQL are the same. But with that said, the processed, aggregated data that gets stored is generally much smaller than large volumes of granular data, so there is much less data to comb through with PipelineDB.

Re:KSQL: Open Source Streaming SQL for Apache Kafka by stuartaxelowen on Tuesday August 29, 2017 @12:28AM (#15122077)
Thanks for explaining!
Re:KSQL: Open Source Streaming SQL for Apache Kafka by sandGorgon on Tuesday August 29, 2017 @06:42AM (#15123368)
this is awesome! will it work on Amazon RDS for postgresql ?