If remembers the state globally so if you leave it open it will always be there when you open a file manager window if you toggle it off it will be hidden next time too.
It's not open by default perhaps because not everyone regards it as maximally useful. Say I open a file manager and have it sized to 1/4 of a 24" 1080p screen. On the left hand side of the screen is a panel which lists common and places and other items. This leaves a space for 15 icons on the screen at 48px size big enough to comfortably read and note some details in a preview.
Opening the information pane reduces that to 10 icons greatly reducing the space available for the main purpose of the application.
What if I open it to the full size of the screen? I can now view 56 icons at 80px where now they actually are a pretty decent size to see whats in them in their own icon.
I think most of the time having the info panel shown would decrease the utility of the window.
Indeed, I don't have any use for information or preview panes, I disable them all when encountering. IMHO they are a distraction from the content itself and make the view arrangement clumsy.