That's my general tendency in giving advice, but I've had people disagree with based on the idea that being a big fish in a small pond can build a student's self-confidence and win the student recommendations from professors and the like. Does that make sense to you?
For me, having great profs who were committed to teaching was really important, but even more important that my classmates were smart and committed to learning. Pretty much everything I learned, I learned by working and talking through things with other students. The professors set the stage, and helped us through the sticking points, but for most of my courses, the 3-4 hours a week I spent with a prof were generally met with an equal or greater amount of time spent in academic interaction with classmates.
I know that there are some people who can't cope when they go from being exceptional in high school, to be being average in a school filled with people who were exceptional in high school. To the extent that this can be predicted ahead of time, I think the right thing is not to steer them to a less rigorous school, but rather to help prepare them to find motivations other than being "the best," and, perhaps, to direct them to an institution that where the tone is more collaborative than competitive.
I would also say you should go to a bigger school. For technical fields, the more students, the more choice you will have for courses. Diversity in a technical field is a great thing because you get to see what you are really passionate about. I ended up really loving applied math after going through an engineering track.