Seems like you could distinguish between "recording" and "listening" in that case. Then "recording" is only initiated upon the activation of a trigger which confirms two-party consent.

Though I wouldn't be surprised if Apple intermittently recorded background noise for ex. training purposes in violation of that, anyways.