I don't see your point, you can judge you own mother untrustworthy. The point is that you need to have an extended and real life contact with someone before deciding to trust them.

Ultimately, the word "fact" in this context means "what someone considers a fact". You can be wrong, but if you can't trust your own senses or judgment, how can you live in this world? Well, rhetorical question, since there are no sincere solipsists.