You could add the whole albums to the playlist, one by one, and then play the playlist w/ shuffle disabled. This last bit makes it a limited workaround, however.
On the other hand, Spotify API could allow to implement the custom solution to shuffle these, similar to https://spotifyshuffler.com/, if you have enough skills, time, and determination to do that.
One workaround that I and other users make use of is to just add the first track of albums as links to the albums themselves. It's whatever. It's a bit stupid this is something I have to fucking deal with in the biggest music interface that the planet makes use of. Anyways, I mainly mentioned that as example of the ethos of Spotify UI/UX and what kinds of user they're primarily catering to. I guess I'm in the excluded group in this implementation of "worse-is-better" or something. (Is there a term for that group? Power user can't cover it because people with disabilities are usually in it.)
That's not the only UX hurdle if you're someone who curates a library of things to listen to instead of engorging from an algorithm. Sure, you can follow albums and artists but since you can't organize them, these features gets useless if you have anything more than 100 items. The UI is in general is way too simple, with very sparse information density compared to your traditional music player. Was it because of necessity or style that all these previous tools opted for all these features Spotify shirks? I'll let you decide that.