Big-O applies to a hell of a lot more than just sorting algorithms, and no language lets you ignore the varying degrees of complexity that come from the choices you make.
Good developers consider Big-O implications all the time, without explicitly thinking about it. They don't ignore it.
We actually test incoming staff to make sure they are aware of complexity by giving them programming tasks that they can hang themselves with by using the wrong data structures or algorithms.
Is there really any good reason not to focus optimization on areas it will make a difference, and guide it with actual profiling results?
O(n) vs O(n^2) is not a small efficiency for non-trivial data sets. This is the point that was being made. Many people forget that the quote hinges on the word "small" and then use that as an excuse to disengage their brain when it comes to basic things.
Taking a quote, out of context really, and using it everywhere as some sort of justification, nay, an unthinking reflex, is, indeed, using it as a crutch.
If you have code which does a database query, parses some result and then generates HTML based on that; this is an algorithm. If your Javascript walks the DOM finding elements then that is an algorithm.
These things all have associated computational costs which will scale depending on the input size.
And chosing the correct data structure comes down to understanding the performances (using Big-O / best-worst-average or Analytic Combinatorics) of the operations you're going to perform and the algorithms you're going to use on these data structures.
It really sucks to have to refactor a codebase because some coder thoughts that lists would have been great there when actually a bi-dir map was what was needed (and vice-versa).