I think technology and science inherently empowers a smaller and smaller number to a greater and greater degree while the size of the playing field remains the same.

It would have been virtually impossible for virtually any percentage of hunter gathers to wipe out humanity let alone the biosphere. Eventually one person crazy enough may engineer all of our doom. This could even be so without malice aforethought.

Once you get to the stage of a planetary civilization expanding further gets fantastically harder. It's entirely possible that most intelligent species go extinct between getting big enough to destroy the requirements for their continued existence and actually escaping their planets.

The more obvious answer of ecological catastrophe is actually a lot harder. We could actually be well on our way to crashing and burning and still not be able to predict well enough that far ahead.