In a language without machine integers exposed to the programmer, there may be no way within that language to see any machine integers. There is a further "way down", but not from within the language.

A proposition about an implementation aspect of the language you're working in is testable in that language if a program in that language can produce output which is different based on whether or not that proposition is true. If the proposition isn't testable in that language, then it doesn't lie on the "all the way down" path which begins inside the language.