You can only tell it's a VM via sidechannel because you know what not-a-VM would do already. In the case of the universe we have nothing to compare to, no way to say "that's an effect of running on the underlying hardware" vs "that's a special case of how the hardware runs". E.g. one could take speculative execution rollback after certain workloads as timing jitter from being in a VM if they didn't know that's just how things work on a CPU and the instructions recently ran have an impact on how the next ones will predict.