Tell that to Einstein.
Einstein's advance was quite spectacular, and early.
Source: Am gravitational experimentalist.
https://www.newscientist.com/article/mg24032022-600-exclusiv...
This is why category theory was not discovered, it was reverse engineered! The reverse engineering steps were:
3. Natural transformations
2. Functors
1. Categories
Edit: Of course, when he said theorist I think he meant people who don't experiment physically.
When we went from 1 coconut -> the set {1}, then we were being really abstract for the times.
But I think your point is that category theory synthesises group theory, linear algrebra, topology, etc. into one concept, which was very much the spirit of the origins of category theory. However, Mac Lane and Eilenberg thought that their diagrams were just an aid to mathematics (much like a Venn diagram, Cayley diagram or a Feynman diagram). But when they realised that natural transformations are so ubiquitous and fundamental, then they realised that their graphs were not just a useful shorthand, but in fact would lead to a whole new type of mathematics. When people thought (not Mac Lane though) category theory was "abstract nonsense" they were making this mistake of thinking that the diagrams are illustrations rather than concrete mathematics.
In the same way, you might thing that {1,2,3} is just an illustration, but in fact it is a rigorous shorthand for a very specific set.
The real meat behind category theory are things like natural transformations and adjunctions. But to get to category theory from there, you do a kind of reverse engineering.