You might be reading too much into the phrase "mathematical fantasies". This is not saying Borel claims they literally do not exist. More that they are not "accessible" or "usable".