And, about that vocabulary point: different Lisp dialects differ in their vocabulary. Common Lisp vocabulary will not get you very far in understanding Scheme. Emacs Lisp vocabulary will serve you only so far if you'e reading Common Lisp. A dedicated Schemer will be tripped up reading EuLisp, and so on. In languages that have a big divide between "built in" and "library", we see this kind of variation at the library level. For instance C is mostly the same on Unix or Windows, but the libraries are mutually foreign languages. `CreateProcess` with umpteen arguments versus `fork` and `exec`, and whatnot. Since C doesn't have built in data structures, everyone rolls their own, creating effectively a different "C with lists" dialect.
Humans do not count parentheses and keep track of nesting levels when reading Lisp, any more than they go through LALR(1) state transitions when reading C++.
We structure the code in 2D, and use visual cheats.
With Lisp, there is no clear distinction between "language" and "library".
In any language, code is hard to understand when it uses a lot of external symbols from a library.
No matter how well you understand C, if I suddenly give you a tarball of the USB subsystem of a kernel you've never worked with, and open a random file, it will look like gobbledygook, even though you "understand" the declarations at the "language level".
Most operators in Lisp code are essentially library symbols; and if you're missing some key symbol (such as the outermost one), you might not be able to understand it at all.
Lisp readability requires not only nice code structure, but a decent working vocabulary: enough vocabulary that you're not tripped up by the standard things that are in the language, and you know what is in the language and what is defined by the program.
For instance if you see some (macrolet ...) form and you have no idea what that means, you may be stuck. What is macrolet? Did the program define that somewhere? An experienced lisper doesn't bat an eyelash. Of course she knows what macrolet is: it's a binding construct for making lexically scoped macros, doesn't everyone?
(But at least you know from the unambiguous syntax and formatting clues what goes with what: what parts of the whole thing are arguments to that macrolet, and which are outside that macrolet! So when you look up macrolet in the reference manual, you know exactly what is being passed to it and can figure it out without any undue difficulties.)