I think it’s usually less of a “I’m not smart enough” and more of a “this may take significant time to understand well.” Topics that are deemed too complex usually just require a significant amount of previous knowledge that you may not already have, and you just need to spend a lot of time thinking and learning about them. More on this here - https://www.benkuhn.net/thinkrealhard/
Sure, it's a nice and egalitarian opinion and all but that doesn't take into account the situation when the guy next to you groks something 10 times faster than you do and can move on to the next topic in the blink of an eye.
As an extreme of this, if it takes you 20 years of hard study to grok a topic that normal people absorb in 6 months ... harsh to say, but you don't fall in the "smart" bucket.
What usually convinces me that we're reaching into the "I'm just not smart enough" territory is when I listen to multiple explanations from multiple sources and I'm still stuck (although ... I find that on certain subject people are just parroting each other a lot, which makes me often question whether the person doing the explaining actually understands the topic at all).