The more precise statement of it is that physics and much of applied math is done in the topos of synthetic differential geometry, where by removing the law of excluded middle from the logic, you can basically make all of this true.

Well, except the Taylor series part. Numerical analysis is perfectly happy to take you to the cleaners even in ultrafinitist settings.