It's actually not uncommon for deaf kids to be forbidden or discouraged, depending on era and local pedagogy, from learning and using sign language. And like any language, it's harder to learn later in life. ASL isn't just English with handwaving. It's a whole other language. Same with all sign languages.