https://en.wikipedia.org/wiki/Red_flag_traffic_laws
These laws didn't last long, fortunately, and ostensibly they were to protect cattle, horses and carriage riders but clearly the idea was to slow the car down to the speed of a pedestrian effectively hobbling it to the point that driving a car didn't make sense.
It's insane that the top leader in accidental death of children and adolescents involves automobiles [1].
Riding buses that are quickly get you where you want to go is not a problem. Riding buses that take a convoluted route and thus make slow progress to where you want to go is misery.
You are correct for the bad state of transit most people have, but good transit is possible and then the things you are worried about are not misery.
Your counter to the above should be building more transit. However in the English speaking world we are incompetent in that these decades, and we refuse to learn from anyone else.
There's plenty of towns and small cities in Europe (specially the "center") that could be considered walk-only and people are pretty happy there, for example Toledo (Spain) or Venice (Italy).
Today in the US, cars are the number one cause of death for children and lag only firearms for accidental death overall. Excusing the daily human slaughter by saying, "Well, they're not _designed_ to kill people like guns are" is not a cunning retort.