There is nothing fancy about the decryption required: https://github.com/plietar/librespot/blob/master/src/audio_d...

Try out librespot if you fancy a comparison to what the Spotify app audio backend could be like (obviously there is no clunky gui here).