Show HN: Pyper – Concurrent Python Made Simple

Re:Show HN: Pyper – Concurrent Python Made Simple by dec0dedab0de on Wednesday January 15, 2025 @11:39AM (#42713226)

I really don’t like overloading pipes like this. I would rather chain methods like how the django orm does it.

you could reassign every line, but it would look nicer with chained functions.

  pipeline = task(get_data, branch=True)
  pipeline = pipeline | task(step1, workers=20)
  pipeline = pipeline |  task(step2, workers=20)
  pipeline = pipeline |  task(step3, workers=20, multiprocess=True)

edit:

I would be tempted to do something like this:

  steps = [task(step1, workers=20),
           task(step2, workers=20),
           task(step3, workers=20, multiprocess=True)]
  pipeline = task(get_data, branch=True)

  for step in steps:
      pipeline =   pipeline.__or__(step)

[ Parent | on HN ]

Re:Show HN: Pyper – Concurrent Python Made Simple by Rickster35 on Wednesday January 15, 2025 @12:06PM (#42713700)

According to the docs, | is syntactic sugar for the .pipe method.

  pipeline = task(get_data, branch=True).pipe(
      task(step1, workers=20)).pipe(
      task(step2, workers=20)).pipe(
      task(step3, workers=20, multiprocess=True))

That's probably chained method approach for those with this preference.

[ Parent | on HN ]

Re:Show HN: Pyper – Concurrent Python Made Simple by me-vs-cat on Thursday January 16, 2025 @07:11PM (#42732599)

This style looks pretty good to me:

    pipeline = task(...)
    pipeline |= task(...)

So does this style:

    steps = [task(...), task(...)]
    pipeline = functools.reduce(operator.or_, steps)

But it appears you can just change "task" to "Task" and then:

    pipeline = pyper.Pipeline([Task(...), Task(...)])

[ Parent | on HN ]