Think of the view of it by an external observer as being 'frozen' at the event horizon as kind of optical illusion caused by the extreme warping of spacetime. For a non-evaporating black hole (as described by general relativity), the 'last' photon coming off it will indeed be at infinity (or with infinite redshift). If (when) the black hole evaporates, the external observer will see that and of course there will be no more photons from the infalling particle.
These two different views are hard to reconcile from our human perspective of how the world works, but they really do come from the mathematics of general relativity.
Here's a really excellent and short video which attempts to explain this in ten minutes. (His explanation of how a Penrose diagram works is fleshed-out in more detail in the Susskind lectures I mentioned above).
PBS Spacetime - What happens at the Event Horizon
Is characterizing the dilation as “infinite” incorrect? I’ve heard that and equivalent statements from people who studied and seem to actually understand this stuff, and it seems correct per the article here.