Avoiding doing something because you don't want to get sued and subjected to a lengthy court battle is completely rational and it doesn't mean that doing that thing is illegal.
For example, for decades many TV shows came up with their own lyrics for Happy Birthday song, even though it was well known that the song wasn't copyrighted, because nobody wanted to get sued and fight that battle. Easier to just change a few words in the script.
It's not just legal issues that'll lead to self-regulation. It's also stuff like this:
Re:Megaface
by devalidating on Tuesday January 03, 2023 @05:56AM
(#34230024)
Simply put, there is less information in music. Copyright infringement cases have been won on as little as a handful of notes. So the likelihood of a copyrightable element from the underlying showing up in the model's output is that much higher. A single melody represents a similar portion of all possible melodies as does a color all possible colors and certainly less than the fraction represented by a single color palette. Yet you cannot copyright a color palette but you can a melody.
Copyright law regarding audio media is entire different beast than visual media; the case law and fair use technicalities do not always line up as simply as "if its legal on visual media it must also be legal on audio."
I mean this is a pretty good point. If they're so sure this is legal, then train on copyrighted audio+video media as they already do with copyrighted visual media.