How would you know? It allows reading privileged data without actually taking over privileged processes, and wouldn’t leave behind any signs of what happened.

It really would depend on what they did and how they did it.

If the "privileged data" they read was, say, the root password which they used to gain root access and then started snooping around the system as root or modifying parts of the filesystem, that could be easily detected.

If the exploit itself was performed over the network or if the privileged data they read was transferred over the network, that might also be detected, depending on how it was sent and where it was sent to.

If they tried to launch attacks or probes from the exploited system they could be detected at well.

Network intrusion detection systems and host detection intrusion systems could both help here.

Not nessaserilly no signs. It involves rep3atadly triggering the error path with specifically constructed values. This could easily leave behind traces in the log from which it can be infered that there was a spectre attack (of course, determining what attack was being conducted, or its succeslevel and information exfiltrated is still difficult)