The desktop metaphor makes it look like the desktop is the starting point. You can understand why someone who has not interacted with a directory through a terminal would think this.