Assuming Nautilus, Ctrl+L. I do wish just clicking on it worked though. I'd guess the reason they haven't implemented that is because it's also the window bar, so it's draggable.