Of course, what you are saying about “stronger systems can prove the answers to the questions of the whether the machines from larger sets, halt” is also true, I just, wouldn’t describe this as “the halting problem”, even though they are I guess basically equivalent.
A typical theory says:
For all x:A, P x
In other words, given any x (from an infinite set A) P x is true. Where P is a proposition.
You typically prove it using straightforward induction/recursion.
And the number of possible states increase so fast that you might as well think of it as infinite.
Gödell showed there are algorithms that we cannot prove, but he did that with self referencing algorithms which we rarely use.
Still, you can’t even in theory prove every interesting property about any problem/algorithm.