There's an additional caveat that may be of interest.

If you execute code in parallel, you may pick up different round-off errors unless you take care to combine the results from parallel tasks in the same order every time.

This can be a problem when simulating non-linear dynamical systems (such as the Solar System).