News for hackers. Stuff that matters.
Re:Faust: Stream Processing for Python by ryanworl on Tuesday July 31, 2018 @05:03PM (#17657106)
I don’t see any discussion of idempotent processing or the Kafka transactions feature in the documentation, and I also see a mention of acking unprocessed messages in the form of acking if an exception is thrown in the application.

These seem like pretty important details to acknowledge up front.

Are there plans to support idempotent processing, or is this already implemented and I just missed it?

The developer facing API looks very nice!

Re:Faust: Stream Processing for Python by dleclere on Tuesday July 31, 2018 @05:21PM (#17657277)
It's hard to say, but Kafka provides a number of important predicates for idempotent processing out of the box for consumers and producers. Given that stream-style-apps are just a series of consumers and producers linked together, there is good reason to believe this would offer idempotent processing.
Re:Faust: Stream Processing for Python by vineetgoel1992 on Tuesday July 31, 2018 @06:41PM (#17657938)
Hi, thanks for the comments. My name is Vineet and I am a co-creator of this project along with Ask.

We do plan on adding support for Kafka exactly once semantics to Faust. As @dleclere pointed out, streaming apps are simply a topology of consumers and producers. We will be adding this as soon as the underlying kafka clients add support for exactly once semantics.