Side channels are really hard to protect against. Caches, buffers (maybe you can check when they are full), branch prediction, sound, vibrations, timing, electricity consumption, em waves, temperature. Things leak all over the place.