I believe this issue is pre-dated even further, back when iPod users didn't like that occasionally a random shuffle would play the same song back-to-back. So the 'random' shuffle was made a bit less random.

CD players inspired the same discussion.

Probably doesn't go back further than that, unless jukeboxes has a random play option.

I think the issue was playing the same artist or album back-to-back. So they made ‘smart shuffle’ in 2005. (https://www.wired.com/story/requiem-for-the-ipod-shuffle/)

A shuffle already implies shuffling like a deck of cards, so you wouldn’t get duplicates unless you had two of the same card, and I that’s how it was described in the manual.

Is there any algorithm that would do that without actually storing a list of indices, or track IDs?

I can't think of why someone would prefer a truly random song being played from a playlist rather than the "deck of cards" method.

It can be biased and still random.

Can you explain that?

A non-uniform probability distribution is still a probability distribution, and, as long as it doesn’t assign probability 1 to any particular outcome, is still “random”.

You don't need to make it less random to achieve that. A 100% random shuffle won't have that issue, you just have to make sure it's actually a shuffle and not something else.

Given a set of songs or albums {a, b, c}

Suppose shuffling results in ordered set [b, a, c]

Suppose user plays through the entire set. Now it's shuffled again before repating.

Suppose the new shuffle results in ordered set [c, a, b]

The user now hears c play after c

I'm going to ignore the other ways to handle that and just point out that that is multiple shuffles and won't be a problem if the user initiates all shuffles.

If the user has to initiate all shuffles, it won't play continuously. If you handle it another way, that demonstrates that it doesn't "just" work with a pure shuffle without using those tricks.

1. Without a clearer spec, we should not assume the user wants any repeats at all.

2. Your claim is not even true as written.

3. I'm tempted to assume the user has a reasonable amount of music, so it will last as long as necessary. And you can't force me not to assume that.

Anecdotally, Spotify shuffle is one of the worst shuffles I've ever used. Or at least it used to be, not sure about now since they added Smart Shuffle. At least used to, maybe still, it would play a lot of songs over and over, but never play others. Like it had maybe 100 songs out of 2000 playing regularly, over and over.

This isn't just me, but all my friends too. We're all the time finding old songs we saved that have never once been played with shuffle, while it's played this one song 3 times in the same day.

Perhaps it doesn't work as well with large playlists? Me and my friends tend toward 1000+ songs in a playlist, but most other playlists I've found are rarely over 250 songs.

Just from reading the article you can see how bad it is, and it makes sense that it would be worse for larger playlists. They've completely discounted the fact that when you play large playlists, you don't listen to the whole thing. The problem with the dithering comparison is that in dithering, you're looking at the whole image at the same time. But nobody listens to a long playlist all at once.

First of all, the core idea that Same Artist == Similar Song and Different Artist == Dissimilar Song is already flawed. There are just way, way more axes than that. Getting 4 slow songs in a row in a playlist of mixed slow & fast dance music is going to feel very bad for everyone.

They give each artist one single "random offset". If this is uniform over [start, end], then it would absolutely kill artists that show up a lot on your playlist, dramatically in favor of artists you have once or twice. If it's from [start, end/N] with N being the number of songs of that artist, it would be a little better, but it's still making the playlist behavior completely different at the start (where it's mostly completely random) from the middle/end (where it's "dithered"). If it's [start, end/M] with M being anything else, it's again dramatically favoring the artists with fewer songs on the playlist.

Instead, they could introduce a "ghost" song for each artist that gets shuffled in and then discarded (or possibly some small constant number of ghosts).

I think you're onto something here. Spotify already saves various metadata about the musical qualities of songs like how "upbeat" it is or how much it "slaps". I'm imagining some kind of K-farthest-neighbors algorithm could make it so that you're constantly being given songs that are as different as possible from the recent ones you've listened to. I dunno, I'm not very well versed in algorithms yet, maybe that would be way too slow.

I don't even understand the goal here. My playlists are built up around topics and themes - most songs in playlists I build are going to be not very different.

I think there's probably a different optimum shuffle experience for real playlists like you describe vs. "all my liked songs" where the latter is often what I'd put on in the car by default.

In the early 1980s I worked in the incoming stock dept of the worlds largest record store (we were physically separated from the actual store). We had an employee controlled music system, mostly playing mix tapes. One of the goals of many of us was to create a never-ending stream of "constantly being given songs that are as different as possible from the recent ones you've listened to"

We were young and not that well versed in the full range of musical expression (yet). Nevertheless, that didn't stop one of us (me? not sure) hitting it out of the park with a 3 part segue from "King of the Swingers" from The Jungle Book soundtrack to the Sex Pistols "Pretty Vacant" to one of the Bach preludes from the WTC. This sort of thing was routine on a daily basis for all of us, and we delighted in the best ones.

Their shuffle is completely broken, but so are all modern streaming music players. It used to be that shuffle would do just that - shuffle the deck of cards (playlist), and then deal the cards in order, never repeating until all cards have been dealt. Now it just keeps the playlist in the same order it was in and jumps all over the place, repeating songs and never playing some. It's very frustrating and woefully broken. Old media players did it correctly.

Well, as long as you don't go too far back: Old hardware media players with shuffle (mp3 players, even my cd player had a shuffle button) often used the same random seed, resulting in the same random order. My guess would be by accident due to restarts not preserving information.

For what it's worth, in the streaming music players category, Apple Music does shuffle as you want it. Probably because it is an old(er) media player, just with a streaming service bolted on. (i.e. if you tell it to shuffle, it'll take the playlist, put it in a random order in your queue, then play through every song once.)

iPod Shuffle is the same.

Agreed; the suggestion that people just expect random shuffle to work differently from that seems well-meaning and rooted in some kind of truth but quite disconnected from the fact you stated. I'm not sure why media players that offer something other than a true "shuffle" can't just provide the option to have either functionality.

I suspect it has something to do with how Spotify makes money and the likelihood they would intentionally bias their service towards playing songs that generate higher revenue per listen. If the complaints are not an example of the bias the article talks about, that is.

That seems like the real problem with Spotify shuffle. The thing that feels un-random about it is that it plays the exact same songs over and over again despite the humongous size of the playlist. It's not that it plays the same artist, or that every once in a while you get a repeat song. Once you get one repeat they're all repeats.

I wish it would randomly pick the next song from the entire playlist whenever it changed songs.

Though as you say it might already be doing this now. I don't remember feeling like I'm rotating through the same 30 or so songs recently. But also I did start working around it, so I may just not be noticing.

> Me and my friends tend toward 1000+ songs in a playlist, but most other playlists I've found are rarely over 250 songs.

Oh it's bad with small playlists as well.

This problem exists even with a smaller playlist. My playlists have about 150 songs each. And every time it plays a certain subset of songs a lot more frequently than others.

This is precisely why I stopped using Spotify. The random was also horrible as you start to "like" songs, those songs continue to show up during random which isn't random at all. Of all of the vast number of songs Spotify claims WTF am I hearing the same ones over and over. If I wanted that, I'd tune into the radio!

I've noticed this too! It's especially bad after using discover weekly and liking a song or two, the shuffle then gets stuck on similar songs. It seems disabling "automix" in the settings returns the shuffle to be more like a traditional random shuffle.

> Perhaps it doesn't work as well with large playlists? Me and my friends tend toward 1000+ songs in a playlist, but most other playlists I've found are rarely over 250 songs.

I think you're right. Recently I spent a bit of time writing about and playing with various shuffling algorithms, trying to see if I could find something that works better for me than the built-in shuffle.

The answer is: it's hard! An algorithm that works for a well-distributed set won't work so well for something with large clusters of similar stuff (think a mixtape vs a playlist consisted of a dozen albums from six different artists). And even when you think you've come up with a good solution, it works well six times, and then the seventh you start to find something that doesn't work quite how you like. It's a process of tweaking.

My final test case was a large playlist like you describe, since I have a few of these, too.

This is what I found worked best for me:

- (Optionally) Fisher-Yates shuffle the whole thing, then - Slice the playlist into X chunks - Shuffle the order of the chunks using Fisher-Yates - Analyze the contents of the chunks - Pick a suitable shuffling algorithm based on the contents - Shuffle the contents of each chunk

There are lots of details around things like, how recently was this artist/album/compilation played? What's the relative tempo or genre of the most recent X songs, and how "harshly" are we willing to change it up? Is this a playlist with a lot of disparate songs (by artist/album/compilation) or is it a collection of like-minded albums and artists? Is the playlist just someone's discography? etc.

It takes a lot of passes and a lot of listening to find and tweak these things to find something that works and feels good. And in the end, I don't think the people who can do this are incentivized to actually do this. Especially considering the linked Spotify article is an entire decade old by now, and the Every Noise at Once guy was let go seemingly at random by Spotify. They care about "good enough for now" more than Actual Quality.

> This isn't just me, but all my friends too. We're all the time finding old songs we saved that have never once been played with shuffle, while it's played this one song 3 times in the same day.

That isn't "shuffle", that's "random".[1]

Normal people know what "shuffle" means. It's only recently (since the iPod?) that tech companies tried to blame the user when the user hit the "shuffle" button and got "random" instead.

[1] When you shuffle a deck of cards and then look at each one in turn, do you expect to see a King of Hearts three times in a row? Why on earth, then, is it acceptable to blame the user when they shuffle a list of songs and then get some songs player twice and some not at all.

Yeah, Spotify's shuffle sucks. I have the same experience with it picking "favorite" songs and choosing to replay them many times while others in the list go unplayed forever. Maybe it believes those songs are my favorites because of how many times they've been played, in a form of circular reasoning.

I think I would be happy with a weighted random selection. Take 2 (or K) random numbers in range [0.0,1.0] and multiply them together to get X. X will favor the low end of the [0.0,1.0] range. Now sort the playlist from least recently played to most recently played. Next song = playlist[round(X * (playlist.Length - 1))].

Something like that. Note that "least recently played" would be a persisted attribute, not something that gets discarded in each shuffle "session". And not even local to this playlist -- just literally what is the last DateTime at which I played that song from my Spotify. It's possible to get a repeat but rare. When I hear a song I enjoy I know I could hear it again soon -- I'm not going to have to work through my entire library to hear it again -- but the odds favor hearing older stuff.

You could also remove the last N songs from the candidate list completely if you wanted to guarantee never repeating super recent tracks. Increasing the K factor would increase the favoritism of long-unplayed songs.

yep spotify has always had this issue which is why it blows my mind that anybody uses it at all. it just takes a very small handful of the songs in a playlist and then shuffles those endlessly, which makes the entire service useless as far as i'm concerned. it's actually just kind of sad, it just isn't a difficult problem to solve, i'm 100% certain one of their junior devs could fix it in an afternoon.

i switched to apple music because of this, which does work properly (unless youre using the web client, in which case you can't click the shuffle button because it only queues up the songs from the first lazy-loaded "page", so you have to scroll all the way to the bottom, play a song from the last "page", then put it into shuffle mode)

youtube music is probably the best option for music streaming if you can stand to give them your money.

There was a dark time with Spotify where ‘random’ played me a couple of random tracks then Cyndi Lauper. Glad that came right.