That would be true if at all times meant 100% all times. But I doubt that this rule exists and is assumed to exist to keep the pilot himself from causing harm. But rather to notice medical conditions as soon as possible. So from this psychological stance it is very likely that there will be moments of maybe no more than a minute of someone being alone in the cockpit - realistically.
As I understand it, FAA regs require that there be two people in the cockpit at all times. So, the Germanwings tragedy would very likely not have happened in the US.