Sure, bio-accumulation is a risk, but the vast majority of meat eaten by humans is low on the food chain. Fish is a whole different story.
If they get fed hi-tox soy/corn crop they may be "low" on the food chain as in "not a predator" but the PTs still accumulate. They do not even live long enough to develop the diseases: we do.
Also they feed animals each other: see mad cow disease.
As an example: it is observed that then some PT got into the environment the small animals did not suffer from it, the slightly bigger animals that eat the small ones also did not, but the larger predators (owls and hawks, iirc) were dying because at their point in the food chain the build up of PTs reached lethal levels.
Like another comment said: dose matters. But I like to add that understanding persistence matters probably just as much.