-- Here, I think you've distilled the basic assumption of the op and I would claim someone with enough exposure to mathematics will know this claim is wrong.

Informal arguments may indeed be carefully and even exactly argued but they nearly always involve implicit emotional appeals, appeals to unstated common assumptions and definitions and so-forth.

Mathematics is different - the arguments are absolute, given the assumptions is can be demonstrated that the conclusions follow, even to the point of the proofs being mechanically verifiable. The reason someone making mathematically assumptions often has to admit their wrong is the arguments have follow a determined series of steps whereas someone with a strongly held, strongly articulated belief-system about the ordinary world can have a plausible answer to every objection but with some portion of those answers being appeals to commonly held beliefs in society.