A good talk along those lines is Feynman on algebra http://www.feynmanlectures.caltech.edu/I_22.html

He starts from numbers 1,2,3 etc and gets to eiθ=cosθ+isinθ which he has as 'This, then, is the unification of algebra and geometry'

I've always wondered if you could go further and deduce our spacetime in a similar manner.