In my experience, we all spend more of our time dealing with what people think things are than what they really are. Even if reality tends to pop its head up now and then.
I think the root comment is correctly describing "how people talk about Big-O" - even though, as you point out, they are mischaracterizing it.
I find this is often a more vexing problem than the underlying performance questions: how do we find good ways to talk about the use (and mis-use) of analysis in a way that produces good tools?
Similarly “competitive programming”, A&D and coding puzzles which I find fun but only tangentially useful, will typically completely neglect engineering constraints. The shape, size frequency and variations of data can almost always be at least estimated in the real world, or simply assumed.
Similarly the term we use is often "N" and "N" could mean anything, and in practice you need to know what "N" you are optimizing for/against. That's where the engineering constraints fit in. It doesn't matter of you pick an algorithm with good Big-O in time (where n represents number of operations, for instance) if your are constrained in space (memory size; cache size; cache locality) and vice versa. Big O notation is an approximation with respect to some function and that n among a selection of choices of different functions/different "ns", that complexity function itself can vary a lot based on your trade-offs. (Which also gets back into Big O is only one such tool, as well. Sometimes you really need to know worst case "Omega notation" and not really Big O notation, if your constraints include lots of worst case things. And so forth.)
Estimation isn't "destiny": know what you are estimating, why you are estimating it, and with respect to what constraints you are estimating it. Big O notation is "back of the envelope math" for algorithms. It serves some great uses, especially in practice, but you need to know what you are estimating and why.
Real assessments of performance go far beyond the depth and breadth of expertise that most students get in undergrad - so it's a fools errand to try and junior engineers about concepts like that. Big-O has a lot of flaws but I get why it seems so useful to all the groups that use it.
It’s really just about making decent estimates, testing and some profiling. There’s of course potential mastery here, specific knowledge and tooling, you can dig down almost indefinitely, but the basic motions can be taught with simple projects. You don’t need Google scale to encounter performance problems and potential improvements.
I know this because I work on small scale things, and had to learn this kind of thing on the job. I can imagine a ton of educational projects that teach this.
Complexity analysis and A&D are very useful when you need them, but they are just two of several tools.
Typically a program is evaluated in terms of something like the "Random-access machine" where every instruction has a constant cost. There's a bunch of additional assumptions hidden in this machine model!
In the real world, the speed of light and the Bekenstein bound conspire to make constant-time random-access to a memory of unlimited size impossible. In practice, a random memory access takes O(sqrt(N)) time. We like to pretend that there's a constant worst-case access time, but that only works out because our machines have a limited amount of memory -- it's not really appropriate for an asymptotic analysis.
So complexity theory based on the "Random-access machine" is just measuring a theoretical instruction count that doesn't necessarily correspond to the real-world run-time. There's other models that get closer, e.g. the "cache-oblivious model".
"People who believe in X are wrong because A, B, C."
"No, X is actually right, because if you know about these obscure parts of X-theory which laypersons never hear about, you will see that it addresses A, B and C."
Of course, both these comments are right, because in addition to the fact that X-theory does in fullness address those issues, few people know how it does.
The courses that introduce big-O are often the weeder courses, they don't aim to education, they aim to flunk out the people that might be hard to teach so that CS departments can gate keep.
When you have top CS researchers having to give talks to remind people that layout matters more than instruction selection, the CS as a whole has focused on the wrong things.
https://www.youtube.com/watch?v=7g1Acy5eGbE
Figure how to build people up, not tear them down.
That is an often re-iterated conspiracy theory. In practice nothing could be further from the truth. Professors have nothing to gain from weeding out students. They have to gain by ensuring a certain standard. By the way it is also better for a student to encounter the difficult parts of their subject as early as possible, so that they can decide whether they have chosen their subject wisely. On the other hand, not failing students on fundamentals means complicating advanced subjects even further --- for everybody.
At some point in life students have to accept that neither their time in university nor their time in the industry will consist only of externally motivating tiny bites of tasks that are awesome. In order to understand complicated matters, you have to put in the work. Spending hours in the gym is also quite boring, but most people accepts that it is necessary for certain results.
I've been involved with undergrad teaching in a University. And the amount of time we spend on finding ways to teach and test better was way more than I could have expected as a student myself. On the other hand I couldn't have imagined the arrogance and sometimes stupidity of first year student claiming to know already what they will need in there career and what not.
So much to unpack there. Some rhetorical hyperbole, other false dichotomy and we are both arguing from anecdote.
Weeder courses exist and as you even admit, they aren't solely meant to educate. That is the point. Anything that is designed to test students resolve as well as not waste professors time is not designed to educate, it is designed to reduce the size and shape of the student population. I applaud that CS departments are adding more and more courses for non-majors. But lets look at the syllabus for a 10 week quarter for the first UW CS course, Foundations of Computing [1] which also expects the students to layout their homework (proofs) in LaTex. Jesus. This is mostly a hazing ritual. In my 3 minute sampling of Reddit, most CS students rank 311 as the hardest course in the major and it is the first. [2]
> At some point in life students have to accept that neither their time in university nor their time in the industry will consist only of externally motivating tiny bites of tasks that are awesome.
I never said anything of the sort, you putting words in my mouth.
[1] https://courses.cs.washington.edu/courses/cse311/21sp/
[2] https://www.reddit.com/r/udub/comments/cdb5kp/cse_311_cse_33...
Your argument might make more sense if 311 were the first CS course, but it's not. You can easily verify that by looking at the course listings which reveals it has a prereq, CSE143, which has a prereq of CSE142. So short of skipping those, it's the third course at the earliest.
311 is the first course of the major and is also by the sample of folks in the reddit discussion also the hardest, covering the most advanced material in the shortest amount of time. It is a weeder course for the major. The 100 level classes are open to everyone.
Also probably a part of a course many people snooze through.
In its own constructed universe, orders of complexity doesn't obey basic physics. Imagine I'm trying to sort a billion unique books alphabetically by content. In order for the books to be unique, m > logn. For many big problems logn < m < sqrt(n) but for a lot of the rest n < m on day one, and stays that way for quite some time. For large objects, an algorithm that avoids comparisons can be faster with real data even if its complexity is n(logn)^2.
Where we fuck up is pretending that mathematical operations are O(1) when they are O(logn) on a good day and O(n) on a bad one. + and - are constant if the numbers fit into a single integer. [] is only constant for n < L1 cache size, and even then only if no other tasks are happening. And as I've just described, < and == are not constant time either. We need better math for these sorts of things.
Nailed it. My frustration is that those words are rarely mentioned, and that atomic isn’t really atomic except in the context of the problem statement and solution.
This is a big misconception about "Big-O". It does not "assume" anything. It's just not what people think it is.
It's an asymptotic upper bound on counting something. In sorting algorithms you count the number of comparisons expressed as a function of collection size. In collection insertion algorithms on some underlying data structure, it's counting some not-very-clearly-defined atomic operations that are being executed when inserting an element.
Where the assumptions come from is if you start using the notation to give performance comparisons and also start assuming things about Big-O that are just not true. But that has nothing to do with "Big-O".
There is a family of symbols, called the Landau symbols, of which Big-O is only one. There is a lot of misuse of the symbol and in many cases what people actually mean is Theta(N) or Omega(N).