If it is such a problem, why not hide the backend behind a nice library instead of a sound server? Say, off the top of my head, libao (after getting Xiph to fix the license), PortAudio, libsoundio, OpenAL and OpenSL come to mind. There are probably more.