Interesting. I never noticed that because I've always used List or Icons as the default view in Finder (Command+1 and Command+2).

Personally, I'd recommend just not using the Column view. It seems pretty janky compared to the other options.

For what it's worth, Command+[ and Command+] will move back/forward between folders, just like Safari and elsewhere.

Holding down Shift with those commands will move between tabs in Finder.