If wikipedia is to be believed, there are something like 60K Taliban. Out of a country of over 30 million. It's pretty easy to recruit that small of a force from that large of a population. What seems more noteworthy is that the Afghan military can't seem to defend effectively against them even with 3x the personnel.