OK, just to cite one example: The population of North Carolina as of 2012 was 9,752,073. 23.7% are under 18 and therefore ineligible to vote. That leaves approximately 7,440,831 eligible voters[1] (I'm ignoring convicted felons and other edge cases for now).

According to the State Board of Elections, there are currently 6,438,531 registered voters. 2,764,322 Democrats, 1,984,251 Republicans, 20,901 Libertarians and 1,669,057 unaffiliated.[2] So that gives us around 1,002,300 potential voters who aren't even registered. So around 13% of the eligible voters aren't even registered and obviously don't vote.

Now that's just one state, and that doesn't even attempt to account for people who are registered, but don't bother to vote, or vote infrequently. But it does suggest that there's a not-insignificant number of people who are sitting on the sidelines during elections.

Edit: Also, as one might expect, there's a Wikipedia page on this topic:

http://en.wikipedia.org/wiki/Voter_turnout_in_the_United_Sta...

[1]: http://quickfacts.census.gov/qfd/states/37000.html

[2]: http://www.ncsbe.gov/