Self reply to mention that one reason it feels so "instant" is that in many cases, it has already auto-completed and started displaying your destination before you finish typing. For instance, I typed "golden gate bridge", and probably by the time I had typed "golden ga" it could start loading, and everything was loaded by the time I finished typing "bridge."
Also, what panoramic viewer is this? Is this something provided by Google, something off the shelf, or something custom written by them?
It is using the standard Google Maps API (version 3).
I suppose one of the things that helps is that this doesn't do the "zoom" effect that looks ugly and slows you down navigating.