In some places, people often walk on the road and it's expected that the drivers will just drive next to them.

If self-driving cars are to use the current infrastructure and drive as a regular car on regular roads then better be able to handle this.