Right, the issue is that nobody checks it and they don't really tell users upfront why they need to. It also notifies you if the number changes later, but seems like that can happen for benign reasons too, so it might go ignored. WhatsApp is similar, and idk if Telegram has this feature.

Someone could orient an E2EE app around these trusted identities, but it'd probably not be very popular.