You're saying a lot of wrong things about CPUs. CPUs do execute instructions "out-of-order", and they do speculative execution and rollback and have another go. Branch prediction is an example.

All of this is possible only with careful bookkeeping of the microarchitectural state. I agree the CPU is a stateful object. But even at the lowest level of interface we have with the CPU, machine code, there are massive gains from moving away from a strict procedural execution to something slightly more declarative. The hardware looks at a window of, say, 10 instructions, deduces the intent, and executes the 10 instructions in a better order, which has the same effect. (And yes, it's hard for me to wrap my head around it, but there is a benefit from doing this dynamically at runtime in addition to whatever compile-time analysis.) In short, it is beneficial to go back and have another go.

This was demonstrated also in https://snr.stanford.edu/salsify/. If you encode a frame of video, but your network is all of the sudden to slow, you might desire to encode that frame at a lower quality. Because these codecs are extremely stateful (that's now temporal compression works), you have to be very careful about managing the state so for can "go back and have another go".

I am less confident about it, but what you say about the universe also seems wrong. What physical laws do you know take the form of specifying the next state in terms of the preceding state? And literally many of them are time-reversible.