Re:Faust: Stream Processing for Python
by dleclere on Tuesday July 31, 2018 @05:21PM
(#17657277)
It's hard to say, but Kafka provides a number of important predicates for idempotent processing out of the box for consumers and producers. Given that stream-style-apps are just a series of consumers and producers linked together, there is good reason to believe this would offer idempotent processing.
[ Parent
| on HN ]
Re:Faust: Stream Processing for Python
by vineetgoel1992 on Tuesday July 31, 2018 @06:41PM
(#17657938)
Hi, thanks for the comments. My name is Vineet and I am a co-creator of this project along with Ask.
We do plan on adding support for Kafka exactly once semantics to Faust. As @dleclere pointed out, streaming apps are simply a topology of consumers and producers. We will be adding this as soon as the underlying kafka clients add support for exactly once semantics.