Trying and failing indeed doesn’t make you stupid but in many circumstances, e.g. being poor, makes you stop trying or the feeling of inferiority seeps in. Also the people around you set a background that make it even harder to escape from.