Re:Mpire: A Python package for easier and faster multiproces...
by flakes on Friday August 11, 2023 @01:02PM
(#37090968)
Depends on the workflow. For one off jobs or client tooling, parallelism makes sense to have rapid user feedback.
For batch pipelines on that work many requests, having a serial workflow has a lot of the advantages you mention. Serial execution makes the load more predictable and makes scaling easier to rationalize.
For batch pipelines on that work many requests, having a serial workflow has a lot of the advantages you mention. Serial execution makes the load more predictable and makes scaling easier to rationalize.