You'll have to take my word for it but I'm under no illusion as to my smartness relative to others. More often than not I'm the dumbest person in the room. I learned the hard way that I have plenty of blind spots and I'm the first person I should be checking for 'sloppy thinking'.
Getting meaning across via words is fundamentally the same process.
-- Here, I think you've distilled the basic assumption of the op and I would claim someone with enough exposure to mathematics will know this claim is wrong.
Informal arguments may indeed be carefully and even exactly argued but they nearly always involve implicit emotional appeals, appeals to unstated common assumptions and definitions and so-forth.
Mathematics is different - the arguments are absolute, given the assumptions is can be demonstrated that the conclusions follow, even to the point of the proofs being mechanically verifiable. The reason someone making mathematically assumptions often has to admit their wrong is the arguments have follow a determined series of steps whereas someone with a strongly held, strongly articulated belief-system about the ordinary world can have a plausible answer to every objection but with some portion of those answers being appeals to commonly held beliefs in society.
The fact that you jumped to 'other people's sloppy thinking' as your first example is somewhat telling. The point is that your whole perspective can shift from thinking that you're smarter than other people and need to correct their mistakes, to understanding how to collaborate with other smart people over difficult problems while everyone is making and correcting mistakes.