If you're interested, https://github.com/vector-im/roadmap/issues/10 is the relevant issue on the public Element roadmap: https://github.com/vector-im/roadmap/projects/1
If someone gives me their phone number I would find it to be extremely rude to them if I just pass it to some third party without their consent. And most people likely do in normal life:
"Hey Bob, can I give your phone number to Alice ? She would like to buy that furniture you re selling."
Yep people will just give their full phonebook to a third party so they can see who else is one the service (and betrayed their thrust by giving it their phone number!). I really hope this is just UX dark patterns, where the apps ask for phone book access but don't explicitly tell users how they are doing this matching. Users doing this willingly would be much worse...