Unlike "neglecting the fact that heat could still pass through the layers", speculative execution doesn't merely allow something that was previously possible. It allows something that was previously impossible, and while doing so undermines another, more critical, feature of the processor.