How about a comparison to modern education? Back before public education gave everyone the knowledge necessary to write for themselves, would it have been imperialist to advocate for that goal? If so, does it matter?