History has never been purely about studying literary records. Thucydides' History includes gratuitous use of oral speeches and discussion of events that predate writing. I can't think of a single modern historian who doesn't make use of archaeological data either.

Equating history with writing is a very anachronistic definition that was popular among Renaissance and early modern historians as a way of legitimizing their preference for classical scholarship over the "dark" middle ages. It's not a good rule of thumb for what is or isn't historical.