Yes. If you don’t want to answer because it might be incriminating, then you have to assert your fifth amendment privileges; you can’t lie and say that you don’t remember.