It's a very weak loose statement, but I think the idea is that leading scientific and mathematical thinkers (think Newton as a quintessential example) were "natural philosophers" who studied whatever caught their interest and took it wherever it went. Astronomers invented lenses and ground them and studied the starts and developed algebra and calculus to model the observations.

Some people were more narrowly focused, like Gauss who did mostly math (but an amazing breadth of math!)

There was a lot of hesitancy about math that couldn't be empirically illustrated by building out of atoms, like irrational numbers and then transcendental numbers and imaginary numbers and then infinite structures.