Right. In the two cities I have personal experience with, the tracks and/or sleepers are still there, just paved over. They are evident (and briefly verifiable) when potholes develop or roadwork excavations go into those areas.

Regardless of whether it's more suited to OSM or OHM, I just think it would be a neat feature to have on Open Railway Map.