To be a bit more future-proof you should give Apache Beam a try. The same code should (theoretically) work with any of the supported runners[0] and so you could deploy it on top of the most suitable framework/technology for your specific workload. Moreover, at this point, the community has several examples of how to add an additional runner.

Edit: I didn't see you mentioned Beam in your second salvo :)

[0]: https://beam.apache.org/documentation/runners/capability-mat...