Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach useful? In common usage, we recognise, as knowledge, various things that cannot be communicated by language, such as knowing how to ride a unicycle on a tightrope (I doubt you can learn it just from a book) and the infamous qualia which supposedly prove that the mind is dualistic. And what about the knowledge of how to use language? How does that get bootstrapped?
This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.