My question was rhetorical. I think we're in agreement - memory accesses may require page-in which requires OS scheduling - but to call the latency of a memory access a type of blocking is sort of confusing/misleading. It may take a long time, and you may get preempted and never return, but those are sorts of different things.
Let's define "may block" first, then, perhaps? What do we mean when we say "network I/O may block"? Usually, this means that the kernel may see your network request and raise you a context switch while it waits for the network response on your behalf. In your last sentence you appear to argue that the reason why the kernel performs a context switch is relevant in determining if an operation "may block", and the GP is arguing that that's a distinction without a difference.
If the definition of "may block" is really just "the kernel may decide to context-switch away from your program", then yes, the GP's assertion that file I/O, memory I/O (mmap) and memory access (swap) are all operations that may block -- the only difference is in degree: from microsecond delays for nvm-backed swap to multi-second delays for network transfers.
Or, of course, I may have misunderstood the GP's train of thought.