That is: If I build my theory on a complicated foundation, then you have to learn the complicated foundation before you can even start to learn my theory.
On the other hand, if I build it on a simple foundation, but that simple foundation means that the theory itself has to jump through a bunch of hoops because the foundation is too simple, that can also make the total (foundation + theory) harder to learn and understand.
So the sweet spot is to use the simplest foundation that does not unduly complicate the theory. (And that may change, depending on target audience.)
Is set theory the best answer? I have no idea, but all of programming in 28 pages, built on a foundation only of set theory, is very impressive.
> but all of programming in 28 pages, built on a foundation only of set theory, is very impressive
Yes, it is :-)
(I don't doubt there are compelling reasons, I just don't know enough about set theory or programming theory to know what they are. Other than the clear benefit of this approach over others in "elementary" educational settings, e.g. US high schools)