News for hackers. Stuff that matters.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by downerending on Tuesday February 18, 2020 @03:06PM (#22359800)
Impressive, but I'd be more impressed if they fixed their random shuffle.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by kossTKR on Tuesday February 18, 2020 @03:51PM (#22360278)
Yeah it's pretty interesting that they undertake this huge task when one of the basic features still doesn't work.

Simply put when you shuffle from all of you liked songs you will mostly get the same tracks over and over - some tracks will stay hidden forever, - pretty weird and annoying.

It seems to stem from issues in relation to this post, ie. sql queries and caching to prevent too much CPU use on their end.

Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by 2bitencryption on Tuesday February 18, 2020 @04:20PM (#22360519)
I think the root cause is because spotify shuffle isn't true "shuffle" in the mathematical, random sense.

They perform some analysis to increase the "perceived randomness" - e.g., if the true random seed picks the same artist twice in a row (totally possible), pick another song by a different artist, or else people will perceive the shuffle as not "random" enough.

Unfortunately I don't have the source for this right now, but I'm sure someone will hop in and provide it if I'm wrong about this :)

Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by claudiulodro on Tuesday February 18, 2020 @05:45PM (#22361228)
They have also further modified the shuffle algorithm within the last year or two to favor putting songs at the top that the user hasn't listened to a lot. There are definitely a variety of heuristics involved with their shuffling algorithm.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by downerending on Tuesday February 18, 2020 @05:16PM (#22360968)
I'm familiar with the idea. Their custom algorithm seems to do the opposite. The order actually being generated has very little perceived randomness, far less than what a true random shuffle would look like.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by sorenjan on Tuesday February 18, 2020 @04:55PM (#22360800)
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by downerending on Tuesday February 18, 2020 @05:17PM (#22360975)
Amusingly, the comments at the bottom are from a large number of others also noting that their algorithm doesn't work as described.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by on_and_off on Tuesday February 18, 2020 @08:59PM (#22362447)
I worked at another music streaming company, we had to do the same.

That was hugely frustrating, but we would get user reports of the random button being buggy when e.g. the user gets 2 tracks of the same album/artist one after the other.

Of course that can happen if we truly randomize your content !

So we switched to a pseudo random algorithm that tries to have consecutive tracks from different album/artists.

Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by nvarsj on Tuesday February 18, 2020 @03:10PM (#22359838)
What's wrong with the spotify shuffle?

edit: Did a search, seems like there's quite a few problems (only playing recently added songs, only playing 100 songs out of the playlist, etc.). I know google music has also had long standing issues with shuffle play - and in fact I left it over these kind of issues. Is it really difficult to implement a shuffle?!

Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by downerending on Tuesday February 18, 2020 @03:29PM (#22360043)
For me, I listen only from "Songs" (my entire collection, which is about 3000 tracks). Even when shuffled, almost everything I hear is something I've heard within the last week or two.

When I use the Amazon app under the same conditions, I often hear a track I haven't heard for a long time. Which is what I'd expect when random sampling from 200 hours of music.

(I don't use playlists, as they're simply too much work.)

Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by fuzzmz on Tuesday February 18, 2020 @03:14PM (#22359873)
It's not really random, in the sense that if you have a playlist and hit shuffle it'll always play in the same order instead of randomizing the play order each time you listen to that playlist. Basically, with the current behavior, once you learned the order of the shuffled songs you can always know what comes next.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by joegahona on Tuesday February 18, 2020 @03:49PM (#22360253)
Is there a technical reason it does this, and why it's so difficult to correct?
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by tjoff on Tuesday February 18, 2020 @04:03PM (#22360384)
Technical debt.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by The_Latecomer on Tuesday February 18, 2020 @03:25PM (#22359999)
Google stopped supporting Play Music a while back to be fair. Have you tried using YouTube music? Would you say you find this same issue there?
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by anoonmoose on Tuesday February 18, 2020 @03:47PM (#22360229)
What do you mean, stopped supporting?
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by reciprocornous on Tuesday February 18, 2020 @04:28PM (#22360595)
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by mrkeen on Tuesday February 18, 2020 @03:19PM (#22359920)
It may be the case that 100 tracks are sent to the device and the shuffle logic chooses from them locally.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by kingosticks on Tuesday February 18, 2020 @04:40PM (#22360687)
Not sure why you are being down voted, this is essentially how Spotify's shuffle works. At least, if you MITM the official client and load a large playlist/context you'll only see a small window worth of tracks being loaded. And you won't see any request from the client when you then shuffle that playlist, it's done locally.

This may, of course, have changed. My experiments while (badly) implementing librespot's shuffle functionality were a few years ago now.

Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by downerending on Tuesday February 18, 2020 @05:19PM (#22361003)
In my case, all of the tracks are already on the device. But yes, it's possible that they're doing something like this anyway.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by sushisource on Tuesday February 18, 2020 @05:12PM (#22360933)
Or the "queue album/song" functionality. It's amazing how absolutely dogshit the Spotify UX is. I keep using it because they have the best selection / device compatability but god the UI is just awful.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by Barrin92 on Tuesday February 18, 2020 @03:26PM (#22360005)
while we're asking for spotify features and in case someone at spotify sees this post: You've put a lot of money into podcasting, please add the 'new episodes' feature of the mobile app to the desktop/web app. Essential feature that's still missing.
Re:How Spotify ran a large Google Dataflow job for Wrapped 2019 by airstrike on Tuesday February 18, 2020 @07:09PM (#22361811)
I'm now convinced it's broken by design. The same songs keep showing up on any song / album radio that's even remotely related, I can't help but conclude, perhaps unsurprisingly, that it's all driven by payola.