Re:Parallel tests. Use all your processor cores to speed up ...
by rb2k_ on Monday January 30, 2012 @05:32PM
(#3530925)
It used to be that parallel tests bundled up the tasks in "groups" and went one group at a time.
e.g. 12 tests on 4 cores ==> 4 groups with 3 tests each.
The problem was that some of the groups might have ended up with tests that take 5 minutes each (finished after 15 minutes) and other groups end up with the 20 second ones (finished after 1 minute). Did that balancing every change?
Another question: How does this work for different RSpec formatters? (specifically: junit xml output or html output). Will it still result in a single file?
Re:Parallel tests. Use all your processor cores to speed up ...
by mahmoudimus on Monday January 30, 2012 @03:19PM
(#3530473)
If you're using nosetests, there's a --processes=NUM option that comes from the multiprocessing nose test plugin, but you should make sure all your tests are safe to parallelize, which, might be very tricky if you didn't think about parallelization before hand.
The problem was that some of the groups might have ended up with tests that take 5 minutes each (finished after 15 minutes) and other groups end up with the 20 second ones (finished after 1 minute). Did that balancing every change?
Another question: How does this work for different RSpec formatters? (specifically: junit xml output or html output). Will it still result in a single file?
In all seriousness though, I think that a look at OS scheduling would provide useful inspiration for test scheduling.