There are Roman structures that remained intact for millennia. I think the main threat to them was people re-using the masonry.

https://en.wikipedia.org/wiki/Pont_du_Gard

Agreed. And Hammam Essalihine/Aquae Flavianae is a Roman bath which is still in use.

Regarding the Pont du Gard, it's intact "due to the importance of its secondary function, as a toll bridge. For centuries the local lords and bishops were responsible for its upkeep, in exchange for the right to levy tolls on travellers using it to cross the river."

I believe bigiain's point is that the structures needed active upkeep to last thousands of years.

Your point is true about people robbing the masonry, but that's closely related to bigiain's comment about "keeping curious humans away from long half life radioactive trash".

So one solution for active upkeep, is to build radioactive waste repositories into toll bridges...

LOL! Yes, that could work. The tricky part would be to keep it from being blown up during some battle, like the Mostar Bridge.