> to what extent is "retain the logical ability" meaningful without attaching it to some knowledge

Some knowledge is obviously required, I'm just less sure that a specific task like coding benefits all that much from having Shakespeare in the training set...