Minor detail, but the undecidability Halting Problem strictly requires an infinite tape (infinite storage space).

In "practice", if you could call it such, a computer with limited memory becomes a "linear bounded automaton" and the Halting Problem is decidable; cf. http://cs.stackexchange.com/q/22925

Of course, big enough memory can mean that it can be impossible to detect termination before the heat death of the universe - we are talking theory here.

actually, the proof for the halting problem is rather esoteric in its demonstration. The claim isn't that you can't predict when most programs will end, but that it's possible to construct a scenario where the program goes out of its way to be unpredictable by running the prediction algorithm on itself and inverting the answer using an infinite loop.