Your comment about the differences in how CPUs deal with floats is very interesting.
I found the following interesting comment, from a GPG employee, talking about how they take care to ensure their float math is deterministic:
"I work at Gas Powered Games and i can tell you first hand that floating point math is deterministic. You just need the same instruction set and compiler and of course the user's processor adhears to the IEEE754 standard, which includes all of our PC and 360 customers. The engine that runs DemiGod, Supreme Commander 1 and 2 rely upon the IEEE754 standard. Not to mention probably all other RTS peer to peer games in the market. As soon as you have a peer to peer network game where each client broadcasts what command they are doing on what 'tick' number and rely on the client computer to figure out the simulation/physical details your going to rely on the determinism of the floating point processor."
For anyone else reading, I'd also suggest reading some of the followup posts such as this one: http://www.box2d.org/forum/viewtopic.php?f=3&t=1800&...
This is news? Isn't this the entire point of the IEEE754 standard, to save us from the days when programs that were well-conditioned on a VAX would become numerically unstable on a Cray?
'Cray instability' sounds like a Star Trek episode but it is apparently something that could happen.
The physics engine is a bit more difficult hurdle I suppose.
The problem with this is that the game needs to be fully deterministic and you'd be surprised how many games use random numbers to help determine outcomes like damage etc.
This can also cause problems when you have complex physics models, many physics engines are not deterministic and small changes between the ways that different CPUs deal with floating point numbers can cause the game to generate a completely different hash.