It's not as black and white as you make it seem. Being in the country unlawfully is not a crime, it's a civil infraction. Entering unlawfully (through something other than a border) is a crime, when it happens, but the subsequent stay is once again a civil infraction.

Most unlawful residents did not enter unlawfully, they overstayed on visas or in some other way went from a legal stay to an unlawful one. These people would not be committing a crime.