Yeah, it's kind of ironic that the modern - and entirely rigorous - direct treatment of infinitesimals comes under the name "nonstandard analysis."
As to the foundations, what Fourier's work shattered was the sufficiency of then established notion of what a function is, which had been limited to what we now call analytic functions; this eventually lead to the abstract definition of a function that we have today.
There's a lot of historic detail in the book that is left out of this snippet, but his point was that mathematicians barely understood the convergence of infinite series and considered an infinite sum of trig functions to sort of be unfathomable. A big part of Cauchy's work made rigorous fourier series.
It wasn't until half a century later, after Cauchy, that mathematicians had a powerful and coherent foundation for calculus. It's true that then interesting ideas such as inginitesimals were rejected because they lacked comparable rigour: was Bressoud conflating these two time periods?