If you only go by statistics and don’t think through the “what if everyone did this” question then giving you the same old content on repeat will have better metrics.
So if I’m watching a video about TIG welding say, to optimise “next click” I suppose I should be recommended mostly more welding videos, with a smattering of engineering and finally a bit of clickbait mixed in case I’m starting to get distracted.
If services are optimising for “keep going right now”, they might be modelling as something like a markov chain of “clicked on next” and that’s how we get stuck in this kind of short-sighted local optimum.
Google Play Music has been the best for me in terms of variety, but it's buggy as hell. It freezes up if it can't play an ad for some reason, won't automatically play when bluetooth is connected (like Pandora does), and other annoying things. (I ride a motorcycle so I can't mess with my phone if something goes wrong)
If a streaming service didn't order automatically generated genre playlists by popularity (with a little randomness) people would stop listening to them and start saying the playlists are full of rubbish songs they've never heard of.
I don't watch TV or have Netflix, but I listen to streaming music, and I noticed something weird. I used to listen to Pandora, and it recommended a lot of different artists that I liked on hearing them for the first time. It single-handedly gave me a taste for 21st century music. But there wasn't enough music in their catalog, so I switched to Spotify. Now Spotify will play more of the songs by people I already like, but very nearly everything it plays that I didn't find on Pandora I hate and it keep playing the same things over and over no matter how quickly and frequently I skip past them.
It's hard for me to believe that they could be this bad and unresponsive unwittingly, but I wonder if perhaps there is some kind of naive algorithm that is failing.
...you may say, why don't I explicitly mark songs "like" or "dislike", but I mostly listen in my car where I can't do that.
One alternative hypothesis is that different people use Pandora vs. Spotify and the ones on Pandora have better taste, but I can't believe that's all of it.