Those 'technicalities of the real world' are the things that lead to new and interesting mathematics. Maybe your 'whole number' concept doesn't properly model different kinds of measurement. Maybe you've got to iterate a couple time on the definition of 'smooth function' before you find one that's useful for modeling actual observations.

Mathematics is not unencumbered by the real world. It's just highly focused on specific classes of encumbrance. (If I put two marbles in an empty sack and poured out three, you can bet mathematicians would be interested in this new kind of arithmetical behavior, and try to model and explain it. Though talking to mathematicians, I get the impression they are so far up in the ivory tower, they think it isn't connected to the ground at the bottom!)

Our programs are not easily factorable. It is hard to separate different interactions across different problem domains. That's a problem of the language, not of the 'purity' of the program. Well, arguably. You could see it the other way, too, I guess.