Re:We Built a Streaming SQL Engine
by necubi on Saturday October 21, 2023 @04:38PM
(#37970241)
I'm with Arroyo — thanks for the mention! I'd be interested to see someone from Epsio chime in with where exactly they're positioning, but you're right that this is (recently) a very crowded space.
I think you can somewhat arbitrarily draw a line between systems like Materialize/RisingWave that are focused on materialized view maintenance (often reading change feeds from your OLTP database) and stream processors like Flink/Arroyo that are focused on supporting more operational use cases and work with a lager variety of sources and sinks.
Epsio seems like it's working primarily in the former mold, with fast incremental computation of materialized views. Unlike Materialize/RisingWave it seems to be designed to run in front of your database, with all queries going through it.
ReadySet is a really cool project in the query caching/materialized view space that I think doesn't get enough attention. Rather than making you define your materialized views ahead of time, it acts as an automated caching layer on top of postgres/mysql that performs incremental computation for components of query graphs.
As someone who's been in the streaming space for years, it's really exciting to see so much energy in the space in the past couple of years after a long period of stagnation!
I think you can somewhat arbitrarily draw a line between systems like Materialize/RisingWave that are focused on materialized view maintenance (often reading change feeds from your OLTP database) and stream processors like Flink/Arroyo that are focused on supporting more operational use cases and work with a lager variety of sources and sinks.
Epsio seems like it's working primarily in the former mold, with fast incremental computation of materialized views. Unlike Materialize/RisingWave it seems to be designed to run in front of your database, with all queries going through it.
ReadySet is a really cool project in the query caching/materialized view space that I think doesn't get enough attention. Rather than making you define your materialized views ahead of time, it acts as an automated caching layer on top of postgres/mysql that performs incremental computation for components of query graphs.
As someone who's been in the streaming space for years, it's really exciting to see so much energy in the space in the past couple of years after a long period of stagnation!