Re:Go package: fanout – make writing parallel code even easier
by sunfmin on Saturday January 10, 2015 @03:56AM
(#8866345)
Thanks, but there will be I/O waiting stuff right? For that whois command example, I find the CPU usage is zero if I am not running it parallel.
Re:Go package: fanout – make writing parallel code even easier
by drtse4 on Saturday January 10, 2015 @04:14AM
(#8866361)
Yup, if your jobs are i/o heavy the cpu time is wasted waiting if you execute them in a strictly sequential way. With your worker pool the cpu intensive part of the jobs is instead being executed concurrently GOMAXPROCS jobs at a time.
Sunfmin, wrong answer, you'd have a 20x improvement with 20 workers only with 20 cores (ignoring the limited overhead), regardless of the number of workers your queue of jobs will be consumed in number_of_jobs*single_job_duration/GOMAXPROCS.