The range of selectors we can actually affect is extremely limited. There's a vast range of selectors we can't affect. In the grand scheme of things, we've hardly made a dent. That's my point.

To back this up, at least this paper claims that the idea that natural selection is being referred inoperative in modern human populations is widely rejected by evolutionary biologists.[1] Changes in human society have actually accelerated selection for certain types of traits that we know of. For example, genes that help people deal with our increasingly unhealthy diet may be strongly selected for.

1. https://www.pnas.org/content/107/suppl_1/1787