Honestly, if you hitched your horse to the side of where they have the shopping carts, or even a lamp post, not only would most not mind, most would think it was cool, and if they really hated it, wouldn't know who was responsible for/had had the right to remove the horse.
I haven't ridden a horse since I was 3 or so, but I rode a steam train through town last Sunday, which would be period accurate with your horse, and people smiled. They waved and took photos and thought it was really cool. Which it was.
It isn't practical, as you say, but it is pretty awesome. And because of that people will accommodate you.
Sure, it might still be technically legal to ride a horse down the street. But soon enough, people started putting in multi-lane highways. Also, stores took out their hitching posts. Then we started designing cities around the car, so what used to be a mile away is now 10. And half of that distance is consumed by parking lots.