You are totally missing the point. A side channel means that someone else on the same host can read your privileged or otherwise inaccessible data. So it is fine if you only use open source and compile everything yourself but what about that other account on the same machine or image? What about that other VM on the same hardware?