Yes I see what you mean. I have tmux mouse mode enabled which makes this possible. This works in nvim as well because it also has mouse support. Presumably iTerm supports selecting text in a single pane even when you have mouse mode turned off.

IMO this setting belongs with the application, and especially with tmux and nvim supporting this natively, you don't really need the terminal itself to implement some potentially fallible heuristic. For example with iTerm if you accidentally drag outside the pane boundary, the selection immediately expands to include both the panes, which is not ideal.