Randomness need not cause anxiety and/or confusion. Think of the sound of rain or waves reaching the beach. As always with the musical experience its an interplay between being familiar with a given sound scape and associating it with pleasurable feelings and the composed sound actually intentionally triggering such responses. My guess is that "aleatoric" composers simply have not struck the right random notes yet :-)