Of course these things are “observable” in the literal sense. And yet, they aren’t considered to be observable by the memory model of any language spec that I know of. Same as CPU power draw, which has been used as a side-channel to extract bits of crypto keys, and is very much influenced by common optimizations.

Practically, if you need to execute a specific sequence of machine instructions in order to prevent side-channel attacks, then you have to rely on assembler, compiler intrinsics and/or OS support. But that was true way before Spectre.