(Professional mathematician here)
Such a conspiracy could never happen. Mathematicians are, in general, scrupulously honest to a fault. There are exceptions of course, but such a scam would need everybody's cooperation.
Moreover, if you have tenure then you would have no incentive to participate in such a fraud (even to the extent of keeping quiet about it). Once you get tenure you are basically working for pride and for the sheer joy of solving problems, and maintaining a big lie would do nothing for either.
No one in bleeding edge science can fully grasp bleeding edge work outside their specialty, but high profIle stuff like this gets a lot of attention.
But then this gets published and other people could point their mistakes. And then it's not only about what the authors got wrong, but also what the reviewers let slip.
And while the number of mathematicians that can understand the proof is low, it is > 1
Because these famous problems were solved, there is now increased demand for Math PhDs with that specialty.
How do you know if the proof is valid, or if they're just scratching each others' backs and pretending they're all brilliant? You might say "Wait, a fraud that big couldn't happen." Just like one of the biggest investment funds couldn't be a Ponzi Scam? (Madoff)
I.e., the few Mathematicians who know enough to be able to check the full proof might be doing the equivalent of this guy:
http://nymag.com/scienceofus/2015/05/how-a-grad-student-unco...