It's really not. Speciation is slow (on the order of 50 generations at minimum), but selection is very powerful, and can cause large phenotypic changes in the span of tens of generations instead.