Baseband firmware is the firmware for the modem. It has no relation to the SoC that runs the phone, unless there is some sort of exploit that allows it direct hardware/memory access. To listen to the mic or capture video, it has to interface with the ADC chip which is not directly connected to it.

There may have been phones in the past that allowed DMA to the SoC from the baseband chip which hypothetically would have allowed a properly crafted exploit on a per app basis, but its not longer the case, as pretty evident by the FBI asking Apple to unlock the phones (if they could access the memory from baseband, they would not need to)