The "broad spectrum" idea is precisely what people are trying to communicate by using the term "neurodiversity". The alternative perspective, which used to be very common in my circles (and frankly is poking through a bit in your sentence about impoliteness), is that human behavior is not diverse and "normal" refers to a narrow, universal standard of behavior that everyone knows and is obligated to follow. A decade or two ago, I would constantly hear people complaining about others who don't listen to "normal" music or have "normal" hobbies.