But how then, do you decide when to drop out? Trying to judge yourself based on the study habits of your peers strikes me an exercise in imperfect information. I would suggest talking to the professor who gave you "emotionally draining" questions and trying to determine what fundamental knowledge you may lack. But perhaps you have more relevant experience you can share.
I think that telling everyone that they can achieve anything they want is doing them a huge disservice. If you talk to actual grad students or post docs, most of them are actually quite unhappy and unsatisfied at their careers. They constantly fell insecure and unstable, and they in fact are so. This is much less common among top ones, because they quite clearly see that they are ahead of others.
In short, if you don’t feel confident that you can succeed in academia/research, leave before the academia chews you out. If you are confident you can succeed, you are probably wrong, but at least you’ll be much healthier psychologically throughout.
And getting into these positions of trust and responsibility should be based on your own definition of "smart enough", correct?