Most early poetry was sung, and we're fine with calling that literature. Calling Dylan's work poetry does step over some normative lines, but they're fairly arbitrary lines.

I know you were being sarcastic, but your comparison isn't really apt. Novels are not a lyric medium: they're prose. Song lyrics are lyrics set to music. Some poetry is "lyrics" without the music (although lyric poetry is not quite the same thing).