I think LLMs are great for summary or pastiche text. They do OK with poetry, but obviously this is a pastiche of existing poetry by necessity, since it can't be rooted in human experience (unless you want poems about being a computer, but that's not what most people have in mind. I think they're great for code, although within rather tight limitations in my experience.

The problem with LLMs for music (as currently implemented, not inherently) is people keep training them on complete tracks. They're very obviously being trained by people who are not musicians.