If one did believe that the disembodied spirits of the dead could be trapped on earth near the place where they died, it's not really much of a additional stretch to postulate that they would be specifically bound to a physical structure, or their pre-death conception of "home."

It's weird to me how so many people's approach to mythbusting is "if ghosts were real, they would behave according to this set of rules I just made up, and they clearly don't, therefore ghosts aren't real." A much better argument would be "There is no evidence whatsoever of ghosts existing, therefore ghosts aren't real." There's no need to try to define ghosts as elementary particles or whatever.