> It doesn't seem that this approach "knows" the actual music. The article doesn't seem to explain how track embedding vectors are produced

That's the thing with transformers, right? It doesn't actually "know" anything about its inputs.

The embeddings are learned (initialized to random).