There's nothing special about the cache hierarchy. It's just one more thing that can be reasoned about using mathematics. If you care about cache locality, you can write down the semantics of an optimizing compiler and prove that this compiler uses the cache hierarchy in an optimal way. Why would that be impossible?
Not only is it possible, but it might even be easier in high-level languages. For example, the argument has even been made that the importance of cache locality is really an argument in favor of higher-level functional languages.
But again, with respect to TLA+, this is all a red herring and completely misses the point. The more important point re: TLA+ is the GIGO/correctness problem. I don't give a damn how cache-efficient my algorithm is if it's not even computing the correct results!
Understanding your hardware is only half the story. To use the hardware effectively, you also need to have mathemaitcal insights into your problem.
A rather extreme proof of this fact is bubble_sort.c vs. quick_sort.java. I think it's clear which of those is using the machine more effectively...
> I doubt you can achieve great performance on a language that focuses entirely on mathematical representations without considering the hardware.
Plenty of very high level languages have extremely effective compilers. SQL query engines come to mind.
> So yeah... you end up with a beautiful mathematical expression ... but runs way slower than an "ugly" equivalent in c code.
TLA+ et al. are about primarily about correctness and designed for situations where correctness is hard to get. Beauty is relevant only in-so-far as it helps one the path to correctness.
But yes, hand-craft code will get you your GIGO much faster :)