Unfortunately I don't think that will work because the attacker is in the middle. They can request the verification number, then forward it to the victim.

It sounds like we want identity verification, which while solved for computers, is much harder for humans.