I believe the 90ms is the window over which the bitrate is constant... If you want to reduce the latency further, you might need to have variable bitrate (eg. perhaps more bits to encode the start of each syllable, and fewer to encode the end of the same syllable).

That in turn adds complexity to other subsystems - typically variable bitrate leads to loss (bad), packet queuing and therefore variable latency (bad), or requires feedback into the video stream (ie. if the audio uses more data, encode the video with fewer bytes, which in turn means encoding must be serial rather than parallel, increasing latency).