The inability to be certain comes from the fact that we always can ask arbitrarily complex questions about the subject matter. To prevent the halting problem, you must prevent complex questions.

Because there will always be statements that are true which the original system is oblivious to, and thus the corresponding questions will be outside its scope (incompleteness theorem).