Part of the difference is the level of competition with the other students. In math at least, if you want to be in classes with and compete against Putnam winners, there are only 4 or 5 schools you can go to do that.