The car didn't replace walking, it replaced carriages, which were easily as heavy and could probably go much faster than the first cars. Sure, the horses would generally try to avoid obstacles, but horses pulling a heavy carriage at speed are not infinitely maneuverable. So, I doubt that cars would have ever seemed like a huge risk initially simply because it wasn't significantly different from what came immediately before it.