It probably happens less often in mathematics, but it is not immune: in 1932, von Neumann "proved" that hidden-variable quantum theories are impossible, but his proof was amazingly flawed (he essentially assumed that an average of sums is equal to the sum of averages), and the flaw wasn't widely noticed until John Stewart Bell pointed it out in 1966 with an anguished quote that has become quite famous:

"Yet the von Neumann proof, when you actually come to grips with it, falls apart in your hands!... It’s not just flawed, it’s silly... You may quote me on that: The proof of von Neumann is not merely false, it is foolish!"

This is more of a mathematical physics example, but everybody in every field could stand to have a bit more humility and rigor.