Absolutely! I have paths in my old neighborhood where if I walk them I can recall specific passages from Ready Player One or Steven King's On Writing. If you need to clear a specific associative memory from a place you enjoy visiting, just keep visiting while listening to other audiobooks and you may lose that specific association.
Kinda problematic if I listen to an unnerving passage in a place I enjoy visiting!