If I were going to guess (not the parent), it's because human parsing is quite localised, and Markov models tend to optimise well for localised coherence.

Their failure mode is long sentences which don't go anywhere (or mean anything), but that's less grating than a blatant parse failure, especially if you're skim reading.

That actually makes a lot of sense, great point.

Claude Shannon[1], information theory guy, worked on machine generated music quite a lot. Some of the ideas transfer. Information theory informs a lot in this area[2].

[1] https://en.wikipedia.org/wiki/Claude_Shannon

[2] http://www.music-cog.ohio-state.edu/Music829D/Notes/Infotheo...

Fascinating!