You don't need to preserve everything to remember history. If we never destroy old buildings, we'd have to abandon all our cities and find new places to build new cities, because we can't just keep living like it's the middle ages (or whatever time period you feel we need to preserve).