There is no way for you to verify whether the caller's phone number is actually the real source. Anyone can impersonate anyone. It's like the source address in an IP packet. You can put whatever you want in there.

This is mostly a US problem though. In Europe, the operators tend to be more strict on this, AFAIK.