A year or so back I stayed over with some extended family, one of which was 'In to IT and earns a living from it. During that time (using the I.T guy's network) I noticed that one of my phones was eating through the battery at a crazy-stupid-mad rate. A quick check showed that this particular phone had begun uploading a metric shit-ton of data on an otherwise normally asymetric data profile (in other words Data Uploaded vs Data Downloaded Ratio).
A check with the 'family IT guy' confirmed that he had Remoted In to a high value clients' system to trouble shoot something at around the same time that one of my phones began to furiously upload any and every thing on its' memory card.
Sounds like one of those "Cool Story Bro" types stories right?
All I will say is that because one of my 'Low Hanging Fruit' thangs got p0wned I was able to alert the $corp of the problem (since the probe originated on their end not mine).
As with all things in Life, YMMV.
Most people have their email, personal photos, medical records, financial records, chats etc. on their phones. If you believe your phone is likely compromised, you should first move all of those off your phone before being concerned about realtime audio recording.