> … video players can load subtitles that are external to the video file but not audio tracks.

Not true for MPV:

    --audio-files=<files>
           Play audio from an external file while viewing a video.

           This is a path list option. See List Options for details.

    --audio-file=<file>
           CLI/config file only alias for --audio-files-append. Each use of this option
           will add a new audio track. The details are similar to how --sub-file works.