I agree with you. I did Spotify for some months and one of the reasons I quit is because I noticed I had stopped listening to music that was not on Spotify. Which is unnacceptable, of course.

Since then I had the music on an SD on my phone. But I'm afraid of losing it and losing all my music, also don't want to have to manually backup the music.

Now I have a Raspberry Pi as a Music Server and it works flawlessly :)

(One SD Cards with the system and music and another one as an automatic system backup so I don't lose any music and can just swap the cards if one fails)