The pollution would be internal to the civilization, not something lasting of import to outsiders. The ability to warp spacetime to the point of violating causality, time travel, might be a death nail for a civilization. Literal prescience could be dangerous. As such technology moved though a society it could be described as a pollution.

Maybe such technology involves the creation of unnatural structures, something like stable wormholes on human scales. If those structures outlived their creators, areas of space containing them could be considered polluted, or at least dangerous to later civilizations.

Douglas Adams did some thinking on this for Hitchhicker's, that time travel caused cultures to stagnate and eventually degrade for lack of motivation.