I'm guessing it's an age gap thing
I'm 35 not sure how old you are but Ive been using Spotify forever and recommendations has been a major part of that UX. And I used Last.fm before that like 40 million other people, which Spotify killed after ending their deal with lastfm and building their own one.
Considering you don't get proper recommendations says enough about how much you use it IMO. You won't get anything back if you don't tell it what you like aka listening to music often.
I'm sure there is a significant segment who loves the discovery like you, and that segment is probably in the millions buy its probably still the minority.
Even in this thread you are the only one I've seen mention this specific aspect.
Most people probably don't care except for a segment of ultra interested people like yourself.
Fwiw YT Music gives me a huge variety of songs it suggests for me at least.
A good example is radio. If you pick a song and do radio Spotify will start to repeat songs after 7 or 8 of them. YT doesn't do it nearly as often.
Ymmv
It sounds like you don't really listen to music if you're only getting the same songs. I don't see how that's possible in any other way, unless youre the one listening to the same small group of songs and it has no data to make useful recommendedations.