I get the impression that this is a relic of how calculus developed. Basically the creators had a vague intuition of an infinitesimal, it was later replaced by supposedly much more rigorous limit definitions. Then it turns out just creating a non-standard system in which infinitesimals exist is perfectly fine, just like imaginary numbers are useful.

The equivalent to i = sqrt(-1) might be something like the nilpotent infinitesimal where epsilon^2 = 0 but epsilon =/= 0.