I highly recommend this document to start, which describes how NginX processes a request: http://nginx.org/en/docs/http/request_processing.html
For getting the mental model right, you have to understand that NginX configuration is declarative, like SQL or PROLOG. That means as opposed to an imperative or procedural language you might be more familiar with, in NginX you describe "what" you want to have handled, not "how" and NginX worries about the "how" under the hood, for the most part.
Of course like any abstraction it sometimes breaks and leaks, but this is practical software we're talking about. One specific big headache for people is when using "if" in NginX, which is a decidedly imperative construct! As such you do have to tread lightly with it; observe If Is Evil: https://www.nginx.com/resources/wiki/start/topics/depth/ifis...
My ideal web server would have a configuration that resembles a decision procedure in a programming language, so I can have a mental model for what makes it choose one route over the others, and how settings are combined, etc.
Somehow Nginx repeatedly makes me and my coworkers very confused—are we crazy or just mildly incompetent? Should I read an Nginx book?
(One time I wanted to do something fairly straightforward, I think using a conditional inside some block, and when I looked in the manual it warned very strongly that this might cause segmentation faults. I was like WTF?)