"Understanding the music" alone isn't helpful unless you know what features are relevant for recommendations, and these must be learned from meta- and usage-data.

Genre, tempo, key, vocalist sound, instruments, and so on. These might all be relevant in different recommendations, at different times, in some particular order depending on the user. The music-content in effect only serves to align tracks along lines in the embedding space.