One slight correction: all of the memory setup is done by the operating system before the program starts at all. An executable file is basically a memory dump -- to start the program, the OS sets up some virtual memory space, copies the program into it(1), and then jumps to a predefined spot in the memory it just copied.

(1) This step is a bit more complicated due to dynamically-linked libraries