Re:Pypeline: A Python library for creating concurrent data p...
by adamcharnock on Monday September 24, 2018 @05:12AM
(#18056110)
This is exactly what I was looking for very recently. Thank you for writing this, I'll certainly look into it.
[ Parent
| on HN ]
Re:Pypeline: A Python library for creating concurrent data p...
by marcyb5st on Monday September 24, 2018 @06:34AM
(#18056493)
What about Apache Beam? Getting started with the Python SDK has been very easy IMHO. Also, you are future proof as you can easily switch runner from Local to Dataflow/Flink/...
Re:Pypeline: A Python library for creating concurrent data p...
by cgarciae on Monday September 24, 2018 @01:24PM
(#18059523)
I use BEAM for my Dataflow jobs. But their local "DirectRunner" is just for testing purposes. As with Spark, BEAM is a huge beast, Pypeline was created with simplicity in mind, its a pure python library, no dependencies.