After data is stored, it can be extracted by attackers who exploit a vulnerability in the OS, or by anyone with physical access who finds out the passcode. (eg. by coercion, by camera surveillance, or by simply looking over someone's shoulder)
If an operating system truly put privacy first, all that data would never be stored in the first place.
Obviously it's a tradeoff. If you want smart recommendations and all the "AI" features, you need to collect and store a lot of data.
If you value privacy above all else, you have to store as little data as possible, and you have to say "no" to features that require analyzing a lot of data.
Would it be _better_ if all Apple software was fully open source and could be independently audited by anyone? Yes. Does that invalidate everything else? No.
Also, Wireshark is a good way to monitor what data your phone is sending to what servers, even if it's incredibly imperfect.
That's not proof of anything, but again, at some point I feel you have to assume good faith. Apple does not have a history of doing stuff like this.
Seriously, the fundamental issue with proprietary is maintenance, ie., not finding but fixing (in a good way) problems and then making those fixes available to other users. Adding features to scratch niche itches is another, though arguably not as critical a matter. But for merely reverse engineering, decompiling, probing memory, fuzzing and all that lack of source code is effectively zero barrier. If it wasn't then source/algorithm obscurity really would be effective for security rather then a bad joke.
Granted, this data is on-device, and we don't know how much (if any) leaves the device, but if Apple really had privacy as their top priority, they wouldn't collect that data in the first place. Privacy is surely important for Apple, but it's not the absolute top.