Re:A dull, resilient stream processor in Go
by techno_modus on Thursday August 16, 2018 @10:00AM
(#17774637)
After reading the section about processors it is not clear whether it can do stateful computations, for example, running sum or moving average. Can I define sliding windows? Can I join two or more streams?
[ Parent
| on HN ]
Re:A dull, resilient stream processor in Go
by jeffail on Thursday August 16, 2018 @10:10AM
(#17774704)
All Benthos processors are stateless, except for a few such as batch, combine, etc, which do basic tasks such as building up batches from consecutive messages. It's possible to create sliding window processors on top of Benthos, but there aren't any general tools for doing that yet.
Re:A dull, resilient stream processor in Go
by asavinov on Thursday August 16, 2018 @10:51AM
(#17775081)
For stateful processing you need something like:
* https://kafka.apache.org/documentation/streams/ - Kafka Streams (and its KTable)
* https://flink.apache.org/ - Flink
* https://spark.apache.org/streaming/ - Spark Streaming
* https://github.com/asavinov/bistro - Bistro Streams
The focus of Benthod seems to be performance and resilience (implemented without persistence).
Re:A dull, resilient stream processor in Go
by spooneybarger on Thursday August 16, 2018 @11:24AM
(#17775419)
Or to toot the horn of the project I work on:
* https://github.com/wallaroolabs/wallaroo - Wallaroo