If the defendant wanted to argue that he no longer remembered the password (but would be willing to decrypt the drive if he could), then that's something that the judge would consider. But a judge can't (and won't) simply say "Oh, maybe he hasn't complied because he just forgot the password. I'm going to let him off"

Witnesses (in the sense of being called to that stand in a courtroom) are frequently asked to tell the court what they saw/heard/did, or from where/whom they received information. Refusing to answer may get them pulled up on contempt charges, and if they claim not to remember then the judge needs to decide whether they are lying.

Given the impossibility of proving (in an absolute sense) that the witness does in fact remember (at that exact moment), it's a game that witnesses are likely to get away with (hence the standard "I don't recall" answer from politicians and beuracrats​), but it's not a universal solution to the "I don't want to tell you" scenario.