News for hackers. Stuff that matters.

A comment on GNU Parallel 2018. Parent comment

Re:GNU Parallel 2018 by hyperpallium on Sunday August 18, 2019 @12:41AM (#20728273)
I don't swear on HN, but Holy Shit, I just added -P to xargs in a testsuite bash script, and it's 7 times faster.

I only have quad core, but -P values higher than 4 give better performance. Maybe because the tests take different amounts of time, and it avoids them backing up. To get x7, I had -P equal to the number of tests.

[ Parent | on HN ]
Re:GNU Parallel 2018 by waterhouse on Monday August 19, 2019 @02:43PM (#20740506)
If the tests are ever waiting for I/O or something, or otherwise using less than one full CPU core, then I think that could explain why running more than the number of cores would be an improvement. If you run "top", or use the "time" utility, what fraction of a CPU core does each test use?
[ Parent | on HN ]
Re:GNU Parallel 2018 by hyperpallium on Tuesday August 20, 2019 @05:48AM (#20745156)
Good thinking, but they don't do any file io, and I'm not sure how it could use less than a full CPU (wouldn't it just finish sooner? Though waiting for io could do that). OTOH it's java (on android, so dex format) and does some classloading of user classes... so maybe that is io? Here's time results:

  real    0m1.818s
  user    0m2.020s
  sys     0m0.900s
top. (It completes in 1 sec, so put in a while loop - BTW top isn't behaving normally, "1" not giving multiple CPUs and not fitting on screen). Jus the top line:

  User 51%, System 23%, IOW 0%, IRQ 0%
[ Parent | on HN ]