This is based on entropy being closely tied to your knowledge of the system. It's one of many useful definitions of entropy.