The thing which would really annoy me about having to use a car everyday and, in total, spending a considerable time of my life driving it, is that you're doing just that, driving.

On public transport, you can read the news, write mails, write docs, read a book, do other things as you please, while still arriving at the location you want to go to, albeit probably a little slower (as you have to wait for the train/bus/tram and possibly have to walk to/from the station etc.).

I have never been to the US, so i cannot comment on the comparative part, though.