Yes. I think many of those features are based on pre-NN feature detectors (such as BPM), and Danceability, Valence and Energy sound like primary components that have been given names.

Echo nest was great for its time, but if they have kept up, they're not exposing their more modern learned features to users anymore.

They were acquired by Spotify, and there's been some work done by/for Spotify since then.

I'm not at liberty to say what, sadly, as I work for Spotify.

I think I can say that one of the main challenges is running this analysis for users. It's prohibitively expensive (or was prohibitively expensive) to use this to keep track of and run recommendations for what users are listening for each user.

It can be used on smaller scales, but, well, it's probably NDA :)

A distributed, local-first architecture much work well for this. I’m happy for my computer to crunch away on my behalf, generating recommendations and indexing stuff. I’m happy to recontribute that work to a common index of some kind.

I def prefer for that common index to have a permissive license though!

Can you say why Spotify's recommendations are so bad? Something like what OP has made should have been relatively simple to make for Spotify for many, many years already, yet that hasn't happen. Is the whole system just rigged to only recommended a few "sponspored" artists?

I doubt that "why does your product suck" is one of the things a Spotify employee is allowed to talk freely about in public!

But I've been watching them, I will speculate. A few years ago, Spotify had two young interns, Sander Dieleman and Aäron van den Oord. We know a bit of what they worked on, because Dieleman blogged on it, and indeed it was something a lot like what OP has made here - only better, I would say. I asked him, and Dieleman was allowed to say that the thing they built was one of the inputs into the then-new Discover Weekly, which made headlines for how outrageously good it was.

But Dieleman and v.d.Oord did not stay at Spotify. They were headhunted by DeepMind, and have had a VERY impressive track record there over the years.

And I wonder why. Was there a conflict between the old school ML of the Echo Nest people and the new fancy neural net kids? Or was it just, as GP alludes to, that the NN methods were just too computationally expensive and they failed to justify their costs to leadership?

Because, as I said above, it's a very complex problem :)

I honestly don't know much about recommendations (and what I know I probably cannot tell). But there's definitely continuous work done on them. But it can also be hampered by extremely conflicting requirements (where "some" both means double-digit procent of users and these "some"s overlap with each other):

- some users want more of the same, some users want a more diverse listening experience. Some of these users are the same user, but on different days

- some users mostly prefer curated suggestions, some users want ranodm stuff. They can also be the same user :)

- some users a heavily weigted to only a few artists, some users listen to evereything and anything. And even this can be the same user :)

- there's probably stuff about licensing, availability, contracts etc. at play as well, because in streaming services it's always there, in very bizarre ways

Basically every single tweak to recommendations will break them. And yeah, Spotify employees will complain about this more than anyone else, all the time :)