Perhaps, but it's also more common than not for affairs to hurt families, and the statement confirms that applied here.

If you have stronger evidence that the statement was untruthful when it said he hurt his family, you may present it.

Until such time, the statement only strengthens the bayesian prior, which is that his affairs hurt his family, and that is what we can conclude.