> The fact that mathematicians cannot devise a method to factor that large product, despite large amounts of money and research time, is what’s valuable!

Well, programmers couldn't devise such a method either. You seem to assume that mathematicians are better than them at finding efficient algorithms, which isn't very plausible.

> But even then your argument breaks down when you consider more complicated cryptographic structures.

Even if help of mathematics is needed there, applied — rather than pure — math might still be sufficient.