If a policy would bring a small benefit to a large number of people, but a large loss to a small number, the small group is much more motivated to fight against the policy, and will often win. This is why it is generally not possible to abolish special programs, shrink the size of the government, etc.

Also note that the "pure flat tax" idea earlier in this discussion and the "universal basic income" idea discussed elsewhere on this website have the same problem: the promised simplification comes from eliminating a bunch of special programs, exemptions, benefits etc., which is impossible.