The reason is a combination of two factors:
(1) Transit will probably always need buses. Rail and subway are great at certain specific things (very dense areas), but they are expensive to build and have trouble achieving good coverage of an entire city.
(2) Buses tend to be big and hold ~50 passengers. Why? Mostly, I believe, because bus drivers have to be paid a salary. You're pushed into choosing large vehicles to try to amortize that cost across more passengers.
What if you could take away that constraint by eliminating the bus driver? Some of those 50-passenger buses could each be replaced with (say) three 16-passenger buses. You'd have a choice. You'd no longer be forced to try to consolidate by reducing the number of routes or the frequency on the routes. Both of these things (long walk to a stop, long wait for a bus) are major reasons why many people don't like riding the bus.
And if you can make transit more convenient for people, you can get more of a critical mass of people using it.
But even in built-up cities, Americans cling to their cars just like they cling to guns and religion. Look at all the uproar in Manhattan, NYC, where they're taking away street parking and putting in protected cycling lanes. People are upset at this "attack" on cars, and somehow think they deserve free places to put their personal vehicles. If you look at another highly built-up city for comparison, Tokyo, they don't have this problem. There is NO street parking; it's all paid lots that cost a fortune. In fact, you're not even allowed to buy a car there unless you can prove you have a place to park it! So most people get around with subways, trains, walking, cycling, and taxis.
The bottom line is that it's cultural: Americans just don't like public transit and don't want to fund it with their tax dollars, but they do want to subsidize car driving with their tax dollars.