"Off The Record" is a technical term to describe a cryptographic scheme for allowing deniable authentication (meaning that only the two parties involved in a communication can be sure the other sent a message - they cannot prove to anyone else that the other person sent them a particular message). The Signal protocol is based off a similar system (Axolotl) which provides similar guarantees.

It has nothing to do with "off the record" check boxes in some UI.