Most multi-threaded code works this way. Even safe languages like Java permit this. I believe the exceptions tend to be interpreted languages like Python, using green threads.
At the assembly level, most modern CPUs are permitted to perform out-of-order execution. I believe the exceptions are pretty rare these days. The custom PowerPC chip in the Xbox 360 guaranteed in-order execution, for instance [0]. GPUs are a different beast.
[0] https://en.wikipedia.org/wiki/Xenon_(processor)#Specificatio...
It does become observable when you try to do synchronization yourself with atomics, without setting appropriate ordering requirements. Lock-free algorithms are usually tricky, and implementing them with minimum ordering requirements makes them even more puzzling.
Depending on access mode, the compiler, or the hardware, can still change the order of operations. It adds whole another level of WTF:
https://llvm.org/docs/Atomics.html#atomic-orderings
> It is also possible to move stores from before an Acquire load or read-modify-write operation to after it, and move non-Acquire loads from before an Acquire operation to after it.