Show HN: Jet – in-memory, fault-tolerant, distributed stream processing

Re:Show HN: Jet – in-memory, fault-tolerant, distributed str... by haxen on Wednesday August 12, 2020 @06:49AM (#24130568)

An Apache Beam Runner is already implemented in Jet: https://beam.apache.org/documentation/runners/jet/

Beam is just an API layer with different backing implementations. But you don't typically use Beam to work with Jet, instead you use its own Pipeline API which is mostly like Java Streams. Jet will also soon get an SQL API.

[ Parent | on HN ]

Re:Show HN: Jet – in-memory, fault-tolerant, distributed str... by netgusto on Wednesday August 12, 2020 @07:14AM (#24130782)

Very cool! Is it possible to mix apis in a single project with Jet Beam Runner? This would make it easier to port Beam projects to Jet, as the migration could be progressive.

[ Parent | on HN ]

Re:Show HN: Jet – in-memory, fault-tolerant, distributed str... by cangencer on Wednesday August 12, 2020 @12:13PM (#24133525)

Do you mean for a single job/pipeline? This wouldn't be possible at the moment. Our current focus has shifted from Beam a little bit - as we found out the beam threading model didn't play nicely at all with Jet's green threads (there is no way to distinguish between blocking and non-blocking calls).

[ Parent | on HN ]