C and Fortran both obey IEEE 754 and have well-defined arithmetic exceptions on most platforms -- but programmers usually instruct the compilers to ignore all that in the interests of faster code.

As for what the differences are, try talking to a compiler engineer. The better ones love how Fortran features like the argument aliasing rule for 100% of code makes generating fast code easier. In fact that's one of my tests for compiler engineers in interviews; if they've worked on a C/C++/Fortran compiler and can't say anything good about the challenges and opportunities of Fortran optimization relative to C/C++, then they haven't really drilled down on performance issues!