You're right. And all of this is nonsense anyway. The people who encourage all this risky behavior aren't going to be there to clean up the mess when you fail and can't pay your bills. And passed up your chance to go to Harvard to do YC.
It pays for them to encourage you to take big risks and encourage big ideas. And they will pick from those risks and if 2 in 10 are a home run then they win big. They are spreading the risk over many ideas (you're only doing one idea) and they are risking other peoples money in general. (Sure they have reputation and other worries but not the same as your loss.)
My guess is, and this is only a guess, that when they evaluate an idea they don't take into account at all what will happen to the founder of the big ambitious idea when it fails. It probably doesn't even enter into the decision to fund the idea or not.
It's like encouraging a friend to go to Hollywood and follow their dream to become a star. If they succeed you have a friend who's a Hollywood star. If not no loss on your part.
It also depends on how long you spent doing the startup and the opportunity cost of doing so. It's one thing if it takes a 6 month chunk out of your life and another thing if it takes a 5 year period. And prevented you from doing something else.
Each circumstance is different of course.
I think it's easy to choose an ambitious startup once you have a success under your belt, and aren't worried about paying the bills (ie. have a small exit/cushion already).
On the other hand, if you still got to pay the bills, and have a family, it makes sense to pursue a smaller idea that doesn't require $10 million in venture capital, or a team of 20 people.
You say that you can start small, and go big.. but I'm not so sure many people here have the luxury on working on any project for 2+ years, and still not have any profit to show for it. It makes more sense once you have some cushion as OP does with his first success.