Re:Resource efficient Thread Pools with Zig
by gefhfff on Monday September 13, 2021 @07:46AM
(#28510257)
Can you explain that Tokio vs crossbeam vs rayon a bit more?
Should I default to rayon or crossbeam because of that?
[ Parent
| on HN ]
Re:Resource efficient Thread Pools with Zig
by ModernMech on Monday September 13, 2021 @10:34AM
(#28512019)
It depends on what you want to do. If you are doing io-bound work, Tokio would be what you want -- you would use it as a runtime for the async capabilities in Rust. If you have cpu-bound work, then rayon is what you want to use. Rayon is a work-stealing parallelism crate -- it will schedule work to be done, and different threads will schedule portions of it as they become available to do work. It's very easy to get 100% CPU utilization across all cores use Rayon if your work is naturally paralellizable, and the interface is dead simple: anywhere you do a .iter(), you can turn it into a .par_iter(), and Rayon will parallelize the work.
Note there is some overhead to using Rayon -- you normally will be better off doing your work on a single thread unless you have a large number of elements in your collection... I found for my application I needed more than 1e6 elements before I saw an appreciable performance benefit to using Rayon.
As others said, Crossbeam is for sending and receiving messages across threads. I use it alongside of tokio and rayon.
Re:Resource efficient Thread Pools with Zig
by YorickPeterse on Monday September 13, 2021 @07:51AM
(#28510301)
Crossbeam is a library that provides various concurrency primitives, such as a queue type that allows stealing of work.
Rayon is a library for running code in parallel. Typically you'll give it a (parallel) iterator of sorts, and it will distribute the work across a pool of threads.
Tokio is a library for async IO operations.
Re:Resource efficient Thread Pools with Zig
by nicoburns on Monday September 13, 2021 @07:52AM
(#28510319)
Tokio is designed for concurrent io, rayon, or tokio-threadpool for concurrent cpu-bound task.
Re:Resource efficient Thread Pools with Zig
by knuthsat on Monday September 13, 2021 @10:41AM
(#28512100)
What about smol?
Re:Resource efficient Thread Pools with Zig
by ModernMech on Monday September 13, 2021 @08:45PM
(#28518814)
smol is in the same category as Tokio - it's a runtime for the async capabilities in Rust.