I've always felt like there was something wrong with how outliers were framed, as though they were wrong, errors, illegitimate, mistakes, noise, etc. They are just, possibly, points from some other process than the one you're interested in.

Seems like a very unscientific way to conveniently ignore something that doesn't fit into the way you want it to be. "Look at all this rigorous math I'm doing! Oh, those points? In my expert opinion they're no good. Just delete them"