Imagine you're living in a culture of subsistence agriculture. And imagine further that you are thin, quiet, nearsighted, physically delicate, and are inclined to spend all your time reading. You are basically unsuited for the available jobs, nobody wants you in a relationship, and what everyday activities are available are ones you don't enjoy. So the village headman decides that you have a disorder and need treatment, by which he means regular beatings. Of course, you might make a fine librarian, but that's not an option.
The same thing was done with gay people for ages. Or women; note how female hysteria was a medical diagnosis that eventually vanished: http://en.wikipedia.org/wiki/Female_hysteria
As another example, quite a lot of American children end up medicated because they are inconvenient for overworked teachers in classrooms that follow industrial models of education. A friend of mine was once a substitute teacher in a class where every single student had been diagnosed with ADHD and was on medication. The daily life that their trait was interfering with is entirely artificial. At some point I think one has to wonder whether the problem is the trait or the constructed environment that the trait isn't a good match for.
We all have deficiencies. What we all need are environments that let us figure out how to cope with them.
Of course, it is all matters of degree and people need to be able to take responsibility for their actions and needs. Not all can and that requires something different.
See also http://intellectualizing.net/2013/09/21/the-trouble-with-ove...
Another issue is that it's sometimes easier to have a personal difference interfere with school than it is to have it interfere with real life. That argument is at http://intellectualizing.net/2015/03/12/schools-are-disorder... Real life is more flexible than most schools.
The problem is lay people think of "autism" the way that creationists say evolution is a "theory". An awkward introvert with technical interests isn't going to be professionally diagnosed as autistic unless they have some pretty serious and defined symptoms. But that doesn't stop misinformed lay-person usage in everyday conversation.