I'd put my money on performance costs.

Personal remove lists basically turn each query from each person with such a list into a completely distinct query, which breaks caching on multiple levels. If a few people are using such a feature or if people sometimes add a couple -sites to a query, no big deal, but if enough people used it with basically unique site lists, the performance degradation would probably make a rollback of the feature inevitable.