It doesn't need to be easy, but unlike with encryption it also doesn't need to be particularly precise. E.g. it's okay to exclude non-poisoned training data because you didn't manage to create a precise enough detector, as long as you don't exclude too much.

Basically any poisoning attack is also fundamentally limited because it needs to be non-invasive enough for humans not to be adversely affected, and that limits the problem space severely - the poisoning mechanism basically becomes reduced to a training mechanism to train out places where the models act different to humans.