I think there are so many side-channels we don't know them all. In the limit I think the classification for side-channels has at least four axes:

1. What data can be leaked? (scope)

2. How difficult is it to construct a gadget?

3. What is the signal-to-noise ratio of the channel?

4. What is the bandwidth of the channel?

The original 3 Spectre variants were basically "whole process or whole of memory, easy, tens of dB, and many kilobytes a second".

If you're looking for binary safety w.r.t side channels, I think modern hardware cannot actually guarantee it.

I can't tell if you are saying "without changing compiler flags, you always lose, even across all possible configurations of even a hypervisor" or if you are saying "without having control over some aspect of the attacker (as in, they can't just give you a binary)". I feel like it can't be the former, or you would have just said that instead of trying to procure some kind of mental framework; but that means the answer now is in some explanation of the latter criteria; essentially, the question is "what are the conservative bounds of current-safe?" (which might get smaller if new vulnerabilities are found or might get bigger if people discover some fascinating mitigation), not "what is a subset of things that are absolute-unsafe"... the latter I can find and even sometimes understand, but the former is what is actually useful for people building systems, so I keep hoping to find a guide somewhere.