Best I can tell, this feature is only applicable if the track is not available in the region you're in, and/or for tracks that are 'local music' that haven't been syncned to the device you're playing on.

If a song is removed from Spotify entirely, it just disappears from the playlist.

I'd love to be proven wrong, however, and start to see all the tracks that have disappeared in the past!