Re:Pypeline: A Python library for creating concurrent data p...
by make3 on Monday September 24, 2018 @11:50AM
(#18058696)
Looks similar to what tf.data does for Tensorflow
[ Parent
| on HN ]
Re:Pypeline: A Python library for creating concurrent data p...
by cgarciae on Monday September 24, 2018 @01:40PM
(#18059663)
Nice to hear that. When I wrote this:
"Pypeline was designed to solve simple medium data tasks that require concurrency and parallelism but where using frameworks like Spark or Dask feel exaggerated or unnatural."
it was actually because I've resorted / hacked into tf.data and Dask in the past just to get concurrency and parallelism. Pypeline is way more natural for pure python stuff.