Yes, the walking movement is one of a newborn's 9 primitive reflexes. [1] However, developing the fine control skills required to balance while walking is a skill that babies learn, and it takes a long time to do so.

[1] https://en.m.wikipedia.org/wiki/Primitive_reflexes