News for hackers. Stuff that matters.
Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @03:21PM (#18232357)
I was interviewed recently. I was asked to implement a solution in Java. It was essentially a stack-based computation engine. The interviewer posed a question and there were enough tacit hints for me to see what he really wanted: an "OO" approach using a type hierarchy + command pattern. He offered unsolicited prompts and I took them: I coupled the command objects to the engine. He liked that.

What we ended up with was O(n) _classes_ (where n = the number of computation operations).

I remained in Java, but took a functional and dynamic approach yielding a three-line implementation of the engine where each command was simply a function. All the tests passed.

I was kind of holding my breath at this point. What will he say? We just went from this huge implementation to three lines. I thought he was going to love it. The crazy simplicity of it all.

But to my draw-dropping surprise he muttered something about adding new types and how he liked that the commands were coupled to the engine.

Is this complexity worship? Or what is this? It's almost as if his brain was refusing to see the solution. Like it couldn't possibly believe that the solution could be that simple. So it just ejected it, put a big black censorship block over it.

I'm currently reading a book on brain hemispheres. Apparently experiments have shown that the left hemisphere will completely and blatantly reject obvious evidence even if the right hemisphere 'has a clear counter proof'. Sometimes I think our industry suffers from an abundance of left hemispheric dominance. Or something like this...

Re:Do We Worship Complexity? by phaedrus on Tuesday October 16, 2018 @04:25PM (#18232922)
A few years ago I needed an emulator for version N of a popular microcontroller. I found an emulator (written in Java) for version N-1 of the same microcontroller. It was written in the style you describe: one derived class per machine instruction type; program to run in emulation is a collection of these via their abstract base class. It was quite pleasant to extend, actually: I simply made classes for the new instructions available in the new version of the microcontroller, added instances of them to the giant table mapping codes to instruction objects, and It Just Worked.

I do note that that is an atypical experience to extending programs in general, and I don't have the alternate, functional approach to compare it to (the emulator was written in a very old version of Java + Swing).

Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @04:28PM (#18232949)
In functional style, you'd probably have a table mapping machine instruction type to a function. These days, even Java (since version 8) supports that, and it roughly compiles down to the same thing (due to how Java implements lambdas).

This case sounds simple, but I still expect the functional approach to save you from a lot of line noise coming from Java class bureaucracy. But in more complex cases, it may be a difference between a couple dozen lines and a couple dozen files.

Re:Do We Worship Complexity? by phaedrus on Tuesday October 16, 2018 @05:28PM (#18233453)
On the other hand, the OOP approach allows you to add printing the function name (for disassembly views), metadata attributes about what flags are affected, clock cycles, etc. You could argue that each of those could be separate tables. Funny how it comes back to the array-of-structs vs struct-of-arrays design question, or maybe I'm just seeing that everywhere because of thinking about Mike Acton's Data Oriented Programming talk.
Re:Do We Worship Complexity? by samontar on Wednesday October 17, 2018 @03:00AM (#18236770)
You get that with Java 8 and later because you can just define a FunctionalInterface and then use a mapping table to function-to-use and it gets really compact. If you want to decorate arbitrary functions that should be fairly trivial too.

Wrote something small like that for a random data generator for an in-house properties tester in Java a short while back. It makes it easier, in my opinion, when you have all the code available in a single screen than if it’s spread out.

Re:Do We Worship Complexity? by iLemming on Wednesday October 17, 2018 @12:14AM (#18236090)
Clojure. Clojure is extremely data oriented.
Re:Do We Worship Complexity? by lasagnaphil on Wednesday October 17, 2018 @02:45AM (#18236707)
I think the most of the data-oriented-programming stuff referenced nowadays are not about conceptual data (how humans understand it), but data in the hardware (how machines understand it). In that regard, in Lisp you can’t really control memory allocation/alignment as flexible as C/C++, (due to its conceptual assumptions with data), so I don’t know how you would do DOP well in Lisp:

(although it might sometimes serve well as a configuration language alongside high-performance C++ code like in gamedev...)

Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @02:53AM (#18236736)
Those are two different kinds of programming, which unfortunately get similar labels.

The C++/gamedev "data-oriented programming" is about structuring your in a way that's friendly to CPU cache. It turns the programmer-friendly abstractions inside-out in order to make execution as efficient as possible.

The Clojure/Lisp/functional "data-oriented programming" is about building your (programmer-level) abstractions around basic data structures like lists and maps. It's meant to simplify your code and make it more robust.

The two "data-oriented programming" paradigms are pretty much opposites in every way except the name. They both focus on data, but the resulting data structures are vastly different.

Re:Do We Worship Complexity? by lispm on Wednesday October 17, 2018 @04:10AM (#18237116)
Actual Lisp implementations need to interface to the C/C++ side, thus they have non-standard / quasi-standard ways to deal with raw/foreign memory and with C types.
Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @02:31AM (#18236628)
I'm... not sure how we went from replacing a function-like object with a function to adding disassembly and clock cycles. Am I missing some context here?
Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @04:51AM (#18237312)
Can't edit or delete anymore so replying to myself. I confused the subthreads and thought this was still the "interview quesiton" part. Obviously the assembly/clock cycles are relevant to the phaedrus's example.
Re:Do We Worship Complexity? by userbinator on Tuesday October 16, 2018 @08:08PM (#18234407)
IMHO the simplest and most straightforward solution for a CPU emulator is a single (or perhaps nested) switch statement. Finding the right case and adding code there is much less annoying than the overhead of creating and updating classes and likely also separate files.

On a related note, this is also why I don't like at all the current C#/Java "modern" style of dozens of files with perhaps one or two actual useful lines of code (the bulk of those files being boilerplate fluff and stupid comments like "// @return the return value".) Debugging is especially irritating since stepping through the code becomes substantially nonlinear with deep call stacks.

Re:Do We Worship Complexity? by hinkley on Wednesday October 17, 2018 @12:13PM (#18240504)
Over the years I’ve heard some good advice on this sport of thing attributed to Bertrand Meyer. I keep meaning to track down citations.

There are conversational styles you can apply to code that make it simple to test (or in his case, make coherent assertions about) that still keep the methods small but cleave it in a way that you can still track in your working memory.

It’s a style I aspire to, and put a great deal of time and energy into. But reading my old code I can say it’s harder than it sounds and it doesn't sound that easy to begin with.

Re:Do We Worship Complexity? by jacobush on Wednesday October 17, 2018 @03:48AM (#18237013)
IMHO a CPU emulator is so simple and well specified, that most any programming style will render a nice solution.

(A CPU must almost by definition must be simple to implement.)

Re:Do We Worship Complexity? by dTal on Wednesday October 17, 2018 @05:44AM (#18237560)
Re:Do We Worship Complexity? by rstuart4133 on Tuesday October 16, 2018 @07:55PM (#18234338)
Interestingly, you managed to illustrate the O(n) debate above pretty well.

O(n) is short hand for here is "what happens when n gets large, but it I'm telling you nothing about when n is small". For example some sorts are O(n log(n)) and some are O(n^2). Naturally for big sorts O(n log(n)) is preferable - but when n is 5 say the O(n^2) will often be faster. People use O(n) precisely because it carries all this unsaid nuance.

In your example you probably have upwards of 30 instructions but possibly hundreds, which sits comfortably in the big side of things. If there was a way to reduce that to an O(1) solution (which implies that if you can add many new instructions without adding lines of code) then you've coded it badly - but we all know that's not possible for a micro controller emulator.

In his example he used O(n), but implied a O(1) solution is possible. The same logic applies. His use of Big-O means for large n, it's possible to add many new cases without adding code. If that is true, his solution is far better than the one wanted by the interviewer. However, it's likely n was small because this code had to be written for a job interview. It's entirely possible he managed to find a enough commonality in those small number of cases to reduce it to being 6 lines, while in the real world problem is indeed O(n). In that case the correct solution depends on the instructions he was given - whether he was asked to solve this particular problem in an optimal fashion, or demonstrate how he would solve the problem in general using those n cases as an illustration.

He doesn't say, so we don't know. His use of O(n) does hint - but the grey beards among us would like to see proof the problem really can be solved with O(1) and the interviewer was demanding an O(n) solution despite that. It does seem like a really weird thing to demand.

Re:Do We Worship Complexity? by dave_sid on Wednesday October 17, 2018 @03:54AM (#18237038)
I think I can explain this. At some point in the past the interviewer has had to get to grips with OO programming. This isn’t easily for everyone, myself included. But eventually it all clicks. And it’s a great thing. Then you start learning about design patterns. You struggle to get to grips with these, you try to memorise them, you don’t quite feel comfortable with them; but one day, it all clicks. And they are great. And you feel a massive sense of accomplishment. In software development, everything moves so fast, but the feeling that you have actually learned something that is almost timeless and will outlast languages and frameworks that come and go every week is a rare feeling. And I really feel that knowledge is priceless. Try then telling them not to bother with all that and write 3 lines of code :-D

If someone has invested a lot of time trying to understand OO and design patterns like this, and had the feeling of accomplishment when it all comes together, then the are going to want to apply them where they can.

I think all that stuff is invaluable knowledge but people shouldn’t fall into the trap of doing something one way just because they have invested the most time in learning how to do it that way.

Also, many software professionals probably find simplicity boring. Simplicity should be taken more seriously as a goal in software development. Sigh.

Re:Do We Worship Complexity? by ripsawridge on Wednesday October 24, 2018 @03:49PM (#18295752)
Wonderful insight!
Re:Do We Worship Complexity? by Apaec on Wednesday October 17, 2018 @01:41AM (#18236386)
What that kind of people need to understand is that "code is a liability, not an asset". Every piece of adhoc code is added maintenance cost, hence the less code the better, no matter the programming language or paradigm.
Re:Do We Worship Complexity? by samontar on Wednesday October 17, 2018 @02:11AM (#18236513)
That’s what I feel too. It’s like in The Goal by Eli Goldratt. The machinery of building product is all cost. Inventory is cost. Machinery is cost. Only finished sold product is revenue.

Analogously, code is cost, infra is cost, only our sold product is revenue.

A thing I did not understand for a long time is the effect this has on value. Something that produces a low value over a long time may well have been value-negative over the period because of the hidden carrying cost.

Re:Do We Worship Complexity? by plopilop on Wednesday October 17, 2018 @05:01AM (#18237361)
I overall agree but I'd like to add some nuance: one-liners are hella compact but no way near maintainable in many cases. For instance 100% of code golf competition results should never be allowed in production.

Also there may be a balance between "some hierarchy in the code so it can evolve" and adhoc code that just works for the current problem. Only having to write a new class for a new feature because everything else is handed smoothly is a bliss.

Re:Do We Worship Complexity? by Apaec on Thursday October 18, 2018 @04:04PM (#18251652)
Wanted to state the main idea first. I definitely agree about one-liners, Perl and APL come to mind.

There are some other aspects to this idea of code economy: correctness and reusability. This definitely depends on the programming language.

In Java(or oop for that matter) you can get correctness but to get reusability you need a whole hierarchy of classes, lots of added code/cost.

In Python you get more reusability out of the box but to get correctness you need to add swaths of unit tests, lots of cost.

In contrast with a functional language such as Haskell you get both with less effort, fp plus a rich type system reduce code enourmosly.

Re:Do We Worship Complexity? by aldanor on Thursday October 25, 2018 @11:16AM (#18301527)
Same with Rust, to some extent -- in some cases it's extremely hard to get a program to compile, fighting with borrow checker along the way, but when it's done and done you can be fairly sure it won't segfault due to a dangling pointer or something stupid of the sort. It also makes it easier for you to trust the code written by others, e.g. when working on a large codebase collaboratively.
Re:Do We Worship Complexity? by Apaec on Friday October 26, 2018 @11:33AM (#18310190)
Rust is another favorite of mine, and with Rust you also get efficiency besides correctness and reusability.

I wish Rust would have gone with an ML syntax though, in particular the Hindley-Milner type signatures allow me to reason better about code reusability, the "fn" and braces just adds noise to me, too much Haskell I guess.

Re:Do We Worship Complexity? by bal00ns on Wednesday October 17, 2018 @12:55PM (#18240986)
I have to agree with this comment. More often than not one-liner rewrites of even trivial functions are unreadable and can't easily be modified, all for the sake of having the appearance of elegance. It's more important to have code that is easily understandable and extendable, which isn't always the case with fewer lines of code.

For example I could write a massive list comprehension in Python in one line that could solve my problem, but it would be much more difficult to read and would be a pain to modify. I'd much rather work with the 10 lines of code it would take to actually spell it out.

Re:Do We Worship Complexity? by netcan on Wednesday October 17, 2018 @05:45AM (#18237563)
There might be some sort of cognitive bias going on here. But, I think the greater driver is more "political."

Complexity creates opportunities for fiefs, little areas only one person understands.

I think we don't recognize what drives a good portion of people, in the tech industry. People pretend to, and are often expected to understand more than they do. Complexity helps us when reporting or manage (up and down). It's harder to grill you, challenge you or commit you to things when everything is complex. Complex requires more people, which is the main scorecard for corporate success.

It's also just unintuitive that simple is harder and better than complicated. Even if you get it, someone else won't.

Re:Do We Worship Complexity? by darkerside on Wednesday October 17, 2018 @07:38AM (#18238034)
And those little fiefdoms, well maintained, can be easy domains to control and reshape according to the needs of relevant stakeholders. Whereas an over simple solution can end up doing one job well but too difficult to modify or extend without rather rewriting it. Depends on the problem you're solving.
Re:Do We Worship Complexity? by i_made_a_booboo on Thursday October 18, 2018 @08:10AM (#18247577)
What led you to this conclusion?

I've come to the same one. I think it explains a great deal of what we see. Even in the hiring process and such. It's actually very primitive at its core. Who's part of who's tribe and who is jockeying for status in that tribe.

Re:Do We Worship Complexity? by phkahler on Tuesday October 16, 2018 @05:31PM (#18233483)
When I was in college one of the candidates for some open position in the engineering school gave an open talk which I attended. He had this whole thing where he researched the right/left brain tendencies of engineering students. He actually had two axis on his pictures but I don't recall what the other axis was so lets keep to the the left/right (logical/creative) axis. He said most students entering engineering were more left brained. He also said retesting people exiting engineering has shifted. Not that the right brains dropped out, but that the same students tested more to the left after an engineering education.

It wasn't clear if that was a shift in thinking or just improving ones skills at a particular way of thinking. In other words, no evidence of any loss just a shift in score on a right/left test. The other notable thing he said is that the right brained people who complete an engineering degree were rather likely to spend some time working in the field and then just change to something completely different.

I remember this well because I tested as a dominant right-brain thinker and have always had this odd feeling that I might want to just stop and do something completely different. Fortunately I've been able to find ways to use creativity in engineering :-)

Re:Do We Worship Complexity? by otabdeveloper2 on Wednesday October 17, 2018 @01:42AM (#18236391)
> left/right (logical/creative) axis

The logical/creative dichotomy is bullshit based on fake science.

In reality the left brain (dominant for most people) prefers sequential, step-by-step thinking, while the right brain prefers simultaneous and spatial thinking.

Math professionals are actually biased towards right brain people. (It's hard to get anywhere in real math using step-by-step algorithms.)

Re:Do We Worship Complexity? by phkahler on Wednesday October 17, 2018 @06:57AM (#18237861)
>> The logical/creative dichotomy is bullshit based on fake science.

>> In reality the left brain (dominant for most people) prefers sequential, step-by-step thinking, while the right brain prefers simultaneous and spatial thinking.

I've never seen anyone make such a direct contradiction so clearly ;-) You define left-brain thinking the same way the "bullshit" does, and then you define the right brain in a way that reflects creativity - at least to me.

Re:Do We Worship Complexity? by deltron3030 on Wednesday October 17, 2018 @05:14AM (#18237418)
>simultaneous and spatial thinking

But can it really be simultaneous (at the same time)? I mean there's still a "micro sequence" involved. it seems that the real skill here is context switching, pattern recognition and applying patterns to different contexts.

Re:Do We Worship Complexity? by Double_a_92 on Wednesday October 17, 2018 @06:21AM (#18237710)
The brain can think about things without you "hearing" it. Then it just mashes everything togheter in a coherent result that you can actively think about.

When I have to think about something hard I try to do it without words in my mind. I believe this makes it easier for the brain to subconsciously work on those, without having to "translate" it first.

However I still use "pictures" in my mind. I wonder if it's possible to actively think about something without using things linked to our physical senses (hearing, seeing, touch,..).

Re:Do We Worship Complexity? by deltron3030 on Wednesday October 17, 2018 @10:09AM (#18239158)
>The brain can think about things without you "hearing" it. Then it just mashes everything togheter in a coherent result that you can actively think about.

Sounds like intuition based on experience, where a new situation is compared against your experience, with the brain filling in the gaps.

>When I have to think about something hard I try to do it without words in my mind.

Hard in the sense of difficult to solve and unfamiliar, or in the sense of the level of focus or concentration?

Re:Do We Worship Complexity? by otabdeveloper2 on Wednesday October 17, 2018 @03:57PM (#18243127)
Probably you're right, yeah.

For example, I know a heavily right-brained person, and when he reads a book, instead of the intuitive sequential pattern (even page-odd page-even-odd-even-odd) he may read it in a zig-zag (even-even-odd-odd-even-even).

The analogy is synchronous processing (left brain) vs multithreaded (right brain).

On the other hand, right-brained people seem to have problems doing 'obvious' things like finishing chores in order, telling left from right or positioning something on a blank page.

Re:Do We Worship Complexity? by Finch2193 on Tuesday October 16, 2018 @10:07PM (#18235395)
Is there really such a thing as left or right brained? Here's [1] an article discussing this. In this talk, what defined a person as "left brained" or "right brained"

1: https://www.health.harvard.edu/blog/right-brainleft-brain-ri...

Re:Do We Worship Complexity? by otabdeveloper2 on Wednesday October 17, 2018 @01:44AM (#18236401)
> Is there really such a thing as left or right brained?

Yes, there really is, but the way it's described in American media is a load of crap. I don't think anybody published actual real science about this in English.

Re:Do We Worship Complexity? by rabidrat on Wednesday October 17, 2018 @01:13AM (#18236290)
Such a left-brained thing to say :)
Re:Do We Worship Complexity? by nicoburns on Wednesday October 17, 2018 @09:14AM (#18238593)
Anecdotally, I've noticed a significant difference in how I think in parts of my life where I was studying philosophy vs parts of my life where I've been doing primarily software development. The whole body pretty much works on a principle of use increasing capability, so it pretty much makes sense...
Re:Do We Worship Complexity? by DanielBMarkham on Wednesday October 17, 2018 @06:16AM (#18237690)
I had a job working for a large Canadian company several years ago.

Coming in, I was told that the specs were difficult, the tech had to be nailed, and they had already tried and failed.

I took a week getting used to the customer, stack, and team. Digging through what they had? They needed two lines of code. It was the same exact situation.

That was one of the more difficult consulting assignments I had. It took me quite a while to gently get the lead architect on-board. I had to position it so it looked like his idea. If I would have pushed hard, he would have pushed back...perhaps getting rid of me and hiring more of a team player.

I've seen this a lot, in a lot of teams, and I think it has its roots in OO/Platonism, although it happens all over. I think the key things most of these teams are missing is that all structure is derivative. That is, there's not a line of code that's in your app that isn't in there for a reason. You're either really careful about that reason....or you add stuff because it looks good or you might need it someday, you can "imagine" how it might be helpful, the guy sitting next to you said you had to have it, and so on.

Re:Do We Worship Complexity? by brightball on Wednesday October 17, 2018 @07:29AM (#18237995)
The longer I've been in programming, the more I've noticed just how much people defend "their way" of solving a problem.

If you do it differently or with a different language, there must be some negativity to latch onto in order to justify the way they did it. The simple ability to say "Oh...cool! Hey Bob, come look at what Jim figured out!" is a rare find.

Re:Do We Worship Complexity? by exabrial on Tuesday October 16, 2018 @04:19PM (#18232870)
I was asked in a Google interview to randomize a List. I wrote Collections.shuffle(list); The interviewer was not pleased.
Re:Do We Worship Complexity? by harryh on Tuesday October 16, 2018 @04:27PM (#18232941)
It's normal to not be pleased when a candidate fails to understand a question (or exhibits smart-ass behavior), though in this case the easy and correct thing for the interviewer to do is to say "OK, how is Collections.shuffle implemented?"
Re:Do We Worship Complexity? by xyrouter on Wednesday October 17, 2018 @12:12AM (#18236080)
> though in this case the easy and correct thing for the interviewer to do is to say "OK, how is Collections.shuffle implemented?"

And very likely that's what really happened. I have been in such interviews and it is sometimes hard to guess whether the interviewer wants the practical answer (calling a library function) from engineering standpoint or the conceptual answer (demonstrating that I can implement the inner workings of the function) from CS standpoint. Often I would just ask a follow up question to clarify exactly at which level of abstraction does the interviewer want my solution to be in?

In situations where I offer a solution that does not match the interviewer's expectation, they clarify the expectations.

Re:Do We Worship Complexity? by exabrial on Tuesday October 16, 2018 @10:53PM (#18235759)
I think it's stupid to ask an interviewee to demonstrate something they should never do at their job.
Re:Do We Worship Complexity? by xyrouter on Wednesday October 17, 2018 @12:03AM (#18236047)
I think it's stupid to assume that a decision regarding time complexity would never come up at their job. Where I work right now, and especially in Google (not where I work right now), it is important that one understands time and space complexity issues well. The data sizes are huge and optimization skills are highly valued.

Sure I can teach constant vs. linear time. But what incentive or reason do I have to spend time teaching these fundamental concepts when I can just hire an engineer who demonstrates the understanding of basic CS at the interview time itself?

Given two candidates with all things equal except that one demonstrates the understanding of CS fundamentals and other does not, why would I want to hire the second person and spend our time teaching him those concepts?

Re:Do We Worship Complexity? by pbhjpbhj on Wednesday October 17, 2018 @07:23AM (#18237972)
>why would I want to hire the second person and ...

Someone commented on here recently that they would take motivated candidates over knowledgeable candidates. That's one reason. I can easily think of at least two others.

There are balance points here that vary according to all sorts of things.

Re:Do We Worship Complexity? by xyrouter on Wednesday October 17, 2018 @02:01PM (#18241705)
I said, "Given two candidates with all things equal except ..."

I mean if there are two candidates who are equally motivated and are more or less equal in all things except that one is strong at CS fundamentals and another isn't, is there a good reason to reject the candidate who is good at CS fundamentals.

In many hiring decisions, I am faced with a similar choice, and I go for the person who is good at CS fundamentals. If two candidates are good in all other ways, then the understanding of CS fundamentals becomes a tie breaker. I see no rationale for selecting the guy who does not demonstrate his strength in CS fundamentals.

Re:Do We Worship Complexity? by zmmmmm on Wednesday October 17, 2018 @01:28AM (#18236329)
I don't really agree ... doing thing A well usually means understanding B, C and D which underpin A. It's actually quite useful to ask people about B, C and D because if you only ask them to do A you don't know if they are doing A the right way because they really know what they are talking about or if they are just cargo culting something they pretend to understand but have no deep knowledge of.
Re:Do We Worship Complexity? by ummonk on Wednesday October 17, 2018 @06:31PM (#18244403)
With most people, it is impossible to gauge how good they are just by observing what they do in half a day on the job. Thus, any interview process must necessarily be somewhat artificial in trying to gain insight into a candidate's abilities in the span of a half a day.

A real life task might be implementing a much more complex library over several days or weeks. Asking the person to implement a simple function like shuffle is the closest you can get in the span of an hour.

Re:Do We Worship Complexity? by Deimorz on Tuesday October 16, 2018 @05:48PM (#18233594)
And on the opposite side, my wife once had a job interview where they asked her to reverse a string. She wrote some code on the board to do it, and the interviewer just dismissed it with something along the lines of "No, that's not right. Try again."

She thought of a different method and wrote it out, and they dismissed it again. After that, she said that she didn't know what they wanted, and they condescendingly told her that she should just use the equivalent of a string.reverse() function (I don't remember which language it was).

Re:Do We Worship Complexity? by Buttons840 on Tuesday October 16, 2018 @06:15PM (#18233758)
Stories like yours and the grandparent comment give the impression that there is little communication actually happening in an interview. I guess if it was an at home assignment given in an email you might not know what the interviewer wanted, but in an actual in-person interview why wouldn't you just say "Well, you could do String.reverse(), is that what you're looking for[?], or would you like me to implement a solution without using String.reverse() or some other way?"

And on the other side, if the interviewers are looking to identify whether a candidate knows about String.reverse() and the candidate starts writing out some 20 line algorithm on the whiteboard, why would you just sit there and watch instead of being like "actually, we're looking for something else, let me ask it a different way; do you know how to reverse a list using the standard library? We're looking for a one-liner here, not an full algorithm implemented on the board."

If your story is true, then apparently there are at least some interviewers willing to just sit there while a candidate writes out 2 completely different algorithms on a whiteboard without interrupting and communicating what they actually want. Such an interviewer is either socially incompetent or trying to make a fool out of the candidate. Either way it doesn't reflect well on the company.

Re:Do We Worship Complexity? by Deimorz on Tuesday October 16, 2018 @07:45PM (#18234279)
Yes, you would think. But for some reason, a lot of interviewers seem to treat interviews as an opportunity for them to prove that they're superior to the person that they're interviewing.
Re:Do We Worship Complexity? by ummonk on Wednesday October 17, 2018 @06:34PM (#18244419)
Yup. If I get even the slightest inkling of this attitude from the interviewers, I will refuse to join the company. I'll give a pass for one smug interviewer, but if I encounter multiple such interviewers, I assume it's not a company I'd like to work at.
Re:Do We Worship Complexity? by tonyedgecombe on Wednesday October 17, 2018 @02:01AM (#18236475)
That or they have already made up their mind and are just looking for an excuse to reject the candidate.
Re:Do We Worship Complexity? by gnulinux on Wednesday October 17, 2018 @01:15AM (#18236297)
When I was interviewing for my current job I was asked to theoretically sort a list with with N elements where N >> M where M is the total number of bytes in RAM (but with infinite disk). I thought ok, easy, I basically explained every step of MergeSort except instead of doing operations in the memory, we need to some operations in the memory (i.e. merging) and then need to dump middle products to disk. You need to be a little more clever since you can't just merge in memory and save in disk (e.g. you can't perform the very last step of merging two N/2 N/2 arrays in memory since N/2>>RAM). Anyway, I was confident with my answer, and my interviewer sounded happy too. Then he asked me "but why don't you use swap?" Hmm I was very surprised. Since this wasn't a coding question (I was already asked dozens of questions involved programming) I thought this was a purely theoretical question. So, I never thought about the systems aspect. On a second glance, it indeed seemed like I just reinvented swapping.

I think CS interview questions can be hard because sometimes you don't know what sort of "model of computation" are you operating on. Am I on a totally abstract setting where all I have is an abstract machine. Or do I literally have an Intel CPU running Linux? Or am I even higher level than that and can think in terms of the abstraction of python. I think sometimes this is not clarified.

One other time in a different interview, I was asked "how does OS free memory in constant time". Having implemented malloc etc a few times I thought this was a stupid question because it depends on the C library implementation of malloc/free as one could also implement free in O(logn) using tree-like structures. Anyway, said something like "it just clears the pointer in the linked list in sbrk()'d space so that node is inaccessible" which apparently was the "correct" answer.

Re:Do We Worship Complexity? by pbhjpbhj on Wednesday October 17, 2018 @07:55AM (#18238094)
>"Or am I even higher level than that and can think in terms of the abstraction of python. I think sometimes this is not clarified." //

Which may be the point, they want you to ask "what am I optimising for", to be aware that there's no simple best answer without needing prompting. Also that the optimisation might be at the business level, like time critical implementation, or use of excess resources gleaned from some other part of the corporation.

So instead of telling you they want a one-liner, you saying "what are our constraints; how long have I got to implement it, ...".

Re:Do We Worship Complexity? by Gravityloss on Thursday October 18, 2018 @07:54AM (#18247493)
Well, that is very informative, but probably not in the way the interviewer intended. Certainly doesn't sound like a great place to work?
Re:Do We Worship Complexity? by kccqzy on Wednesday October 17, 2018 @01:01AM (#18236254)
It's clear to me that in a tech interview, unless otherwise told, it is your algorithm skill that is being tested. If I were you, I would begin talking about Fisher Yates, outline its implementation, explain why it correctly shuffles the list, show why it's an O(n) solution, etc. I might even talk about alternatives.

It's always better to demonstrate in an interview that you have a lot of knowledge about whatever that is being asked. It's not just being able to code, but also the ability to explain why and how it works.

Re:Do We Worship Complexity? by flukus on Wednesday October 17, 2018 @01:18AM (#18236309)
Sometimes tech interviews will ask questions that are more along the lines of API memorization. They can be valid questions too, I've worked with "senior" developers that would implement their own sort algorithms (in .net) rather than use the built in ones. "Does this enormous framework I'm using have a way to do x" is a question some developers never ask.

It all comes down to the question asked, if they were literally asked to randomize a list then OP gave the correct answer, if they were asked to implement a list randomizer they were wrong. If the interviewee didn't make this explicit they were wrong.

Re:Do We Worship Complexity? by humbleMouse on Wednesday October 17, 2018 @02:59AM (#18236763)
This is completely subjective and depends on who is interviewing you. I could give a crap about someone's deep algorithim knowledge. I want to see them write productionizeable code.

Granted, there should be 1-3 people around who can be tapped for algo knowledge when bottlenecks need to be addressed. However the rest of the dev team just needs to worry about productionizeable code...

Re:Do We Worship Complexity? by mrkipling on Wednesday October 17, 2018 @05:05AM (#18237373)
> I could give a crap about someone's deep algorithim knowledge.

That's just proving the point of the person that you replied to, though. If you could give a crap then you obviously care about it.

Re:Do We Worship Complexity? by andy_ppp on Wednesday October 17, 2018 @03:12AM (#18236828)
Often the people who know the algorithms struggle to write maintainable code that isn’t esoteric garbage to everyone but them and will fight to the death to say their way is correct. So your mileages may vary...
Re:Do We Worship Complexity? by rcfox on Tuesday October 16, 2018 @04:32PM (#18232980)
That's a reasonable answer.

Of course, the follow-up question is going to be: Randomize a list that doesn't fit into memory.

Re:Do We Worship Complexity? by err0ne on Tuesday October 16, 2018 @11:09PM (#18235828)
I would have failed you as well. The point is to test whether you have a clear picture of what goes on one layer of abstraction below or if you are confined to the surface.

I don't get the fuss about interviews. They seem to largely consist of basic programming exercises.

You don't need to know the solution before hand and it is easily intuited on-the-fly. I would never hold it against someone to miss some corner-cases or maybe go for a naive implementation first.

I remember a few years ago, someone was complaining that they had been rejected for not being able to reverse a binary tree even though they had a copious amount of OSS.

The thought process is simple and it's an exercise that students do within the first few weeks of their freshman year.

    (struct node (value left right))

    (define (reverse-tree root)
      (cond
        [(equal? root 'EMPTY) root]
        [else
             (node (node-value root)
               (reverse-tree (node-right root))
               (reverse-tree (node-left root)))]))
    
A tree is intuitively defined as a recursive data-structure.

There is one base case: when we reach a leaf.

We want to reverse the left and right subtree at every stage of recursion.

Combine all of this and it's done. I would even be content with a pseudo-code implementation.

Re:Do We Worship Complexity? by vvanders on Wednesday October 17, 2018 @12:10AM (#18236071)
I would have given a +1 for creativity. If the interviewer didn't say that standard libraries were off bounds that's fair game IMO.

I've asked similar questions and if I forget all the constraints just a simple, hah that's clever + reframe the problem is a reasonable approach and doesn't come off as being a jerk(keep in mind the candidate is interviewing you/company as well).

Re:Do We Worship Complexity? by err0ne on Wednesday October 17, 2018 @12:43PM (#18240861)
"Implement a method that shuffles a list" vs. "Shuffle a list"

I would want the candidate to ask me whether they can use their language's standard library. I would say no/yes depending on how the question is framed.

The absolute worst is when question is framed like 1) - the candidate just makes an unwanted assumption, and then acts outraged or smug when I tell them I expect them to demonstrate that they aren't oblivious to the underlying implementation.

Then we venture off whether they can make improvements/trade-offs if the list becomes 10K, 1MM, 1B, etc. records of N bytes etc.

Re:Do We Worship Complexity? by exabrial on Tuesday October 16, 2018 @11:15PM (#18235857)
I'm much more concerned if the candidate is a person that shows up on time, with a good attitude, takes heat and shares credit, builds people up and isn't afraid to speak their mind. If they have a rudimentary understanding of the time constraints of linear vs constant time that's a bonus, otherwise I could teach them that.
Re:Do We Worship Complexity? by xyrouter on Tuesday October 16, 2018 @11:58PM (#18236027)
I come across candidate A who shows up on time, with a good attitude, takes heat and shares credit, builds people up, isn't afraid to speak their mind and offers a smart-ass answer to a problem designed to evaluate whether the candidate understands the underlying concepts.

Then I come across candidate B who shosw up on time, with a good attitude, takes heat and shares credit, builds people up, isn't afraid to speak their mind and offers a genuine and insightful answer to a problem designed to evaluate whether the candidate understands the underlying concepts.

I am definitely going to hire candidate B.

Re:Do We Worship Complexity? by sgift on Wednesday October 17, 2018 @03:13AM (#18236830)
And where did they give a "smart-ass answer"? A question was asked, an answer was provided. Does the answer solve the problem? Yes. If you want to know more then ask. If you want to hear something else that's a failure of the interviewer, not the interviewee.
Re:Do We Worship Complexity? by BurningFrog on Wednesday October 17, 2018 @12:54AM (#18236219)
But all this amounts to trivia questions about things you would never actually do or have to know when working as a software engineer.

The point of a library function is that I don't have to worry about how it works, so I can focus on writing things that are not already in libraries!

Re:Do We Worship Complexity? by flukus on Wednesday October 17, 2018 @02:51AM (#18236729)
> The point of a library function is that I don't have to worry about how it works, so I can focus on writing things that are not already in libraries!

The point of a library function is so it doesn't have to be re-implemented everywhere, not so you can be oblivious to how it works. If your oblivious to what is going on behind every function call you will write terrible code.

Re:Do We Worship Complexity? by anon4242 on Tuesday October 16, 2018 @06:44PM (#18233934)
I was asked in a Spotify interview to reverse a string as a warm up. When I wrote std::string(s.rbegin(), s.rend()) the interviewers seemed pleasantly surprised.
Re:Do We Worship Complexity? by nambit on Tuesday October 16, 2018 @04:21PM (#18232890)
To be fair, using a library to do what you want is not what they're asking for. This is completely different from the functional vs OO discussion above.
Re:Do We Worship Complexity? by exabrial on Tuesday October 16, 2018 @04:23PM (#18232910)
On the contrary, if someone that worked for me needed to randomize a List and implemented their own function, I would be gravely concerned about the rest of work and what they're spending their time on.
Re:Do We Worship Complexity? by ummonk on Tuesday October 16, 2018 @05:15PM (#18233341)
And when Google implemented Java for Android, how, pray tell, should they have implemented Collections.shuffle?
Re:Do We Worship Complexity? by uryga on Tuesday October 16, 2018 @05:29PM (#18233463)
Implementing a stdlib isn't a thing you do every day. Presumably GP was referring to more common situations, where using the stdlib is (usually) preferable to rolling your own solution.
Re:Do We Worship Complexity? by anyfoo on Tuesday October 16, 2018 @09:46PM (#18235083)
They were interviewing for Google. It's not uncommon to have to implement a standard library function (or not be able to use one) in many contexts at Google. Besides, the question is just the most basic setup to assess how a candidate would fare in comparable situations.
Re:Do We Worship Complexity? by gaius on Wednesday October 17, 2018 @03:36AM (#18236955)
They were interviewing for Google.

99% of developers at Google are doing commodity work. It is are practically IBM or HP now.

Re:Do We Worship Complexity? by XorNot on Wednesday October 17, 2018 @02:50AM (#18236723)
Is it though? Because the answer to "suddenly I need to implement a standard library" is "I google the accepted solutions" and if I don't have Google then I goto the library and check out a bunch of old CS books.
Re:Do We Worship Complexity? by ummonk on Wednesday October 17, 2018 @11:39AM (#18240106)
And then the interviewer's follow-up question should be "fine, since you don't want to indulge me and implement a solution to this toy problem, I'll ask you to implement a much more complicated function which doesn't exist in any library!"
Re:Do We Worship Complexity? by monocasa on Tuesday October 16, 2018 @05:58PM (#18233658)
They grabbed it out of harmony.
Re:Do We Worship Complexity? by anyfoo on Tuesday October 16, 2018 @09:43PM (#18235040)
But you would not be concerned about a candidate that does not understand that this question is clearly (very clearly) meant to assess how they understand and approach algorithms and randomness?

It's obvious that you should use a library function if there is a library function.

Re:Do We Worship Complexity? by pbhjpbhj on Wednesday October 17, 2018 @08:54AM (#18238456)
A question like "how do you reverse a string" surely is like asking a chef to cook an egg? Any idiot can do it, it's got to be an opening on to "who are my customers?", "what are the cost and time constraints?", "what team members do I have?" "what's our angle" ...? If the answer is just "boil it for 5 minutes" then you know you're looking at someone with simplistic thinking; which is fine if you just need someone to mindlessly do as they're told. They might be an absolute expert at it.
Re:Do We Worship Complexity? by anyfoo on Wednesday October 17, 2018 @04:23PM (#18243400)
Yeah, it's just an opener. Setting the stage, making sure everyone is on the same page, and then you can build onto that à la "and now suppose xyz...".
Re:Do We Worship Complexity? by ummonk on Tuesday October 16, 2018 @05:18PM (#18233361)
This sounds like a good example of everything wrong with the cult of Java programming / design patterns.
Re:Do We Worship Complexity? by tralarpa on Wednesday October 17, 2018 @04:05AM (#18237082)
Does it? The class-based solution looks very convenient to me for large projects. Let's say I am new to the project and I don't know what commands exist. I just ask the IDE to show me all sub classes of the command super class. That sounds much better than searching the functions implementing the different operations everywhere in the source code. Not everything that is easier to write is easier to read two years later (even if it has much less lines).
Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @06:50AM (#18237828)
Unless there's some magic involved (like pulling code from strings at runtime, via reflection), you'll have a place in which all those functions show up together. A central table, a switch statement in a factory method, or something. You'll have it in either the class-based or function-based implementation.
Re:Do We Worship Complexity? by tralarpa on Wednesday October 17, 2018 @08:05AM (#18238146)
> you'll have a place in which all those functions show up

I don't see why. For example, imagine a graphical editor. GUI events will trigger some actions and those actions will trigger some commands being execute (and placed in an undo buffer etc.). I don't see where that central place should be and why you would need it at all.

And now imagine a large project with different ways to execute commands ("execute", "executeWithUndo", "executeInTransaction",...), people composing new commands from existing ones, etc. Soon grep becomes your best friend. Or just press Ctrl-H or whatever in your favorite Java IDE to see the class hierarchy.

Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @11:11AM (#18239815)
> and why you would need it at all.

Because in the functional model, you're not abusing the filesystem or inheritance hierarchy to do your bookkeeping.

Also, I said functions, not necessarily anonymous ones. There are ways to solve this use case. You can stash your actions as named public static functions in a "Commands" class. Or you can create one dummy class with @Functional on its only method, that essentially forwards the call. So in your UI handlers, you do button.setOnAction(new Command(someFunctionSomewhere));, instead of having a separate class for every possible action. You can now find all actions by looking up calls to Command c-tor(s), and you can build subtypes of Command as you need some special functionality. Note that this way, you don't commit yourself early to a huge type hierarchy.

In my experience, even in large projects, Command objects tend to sit in the filesystem, wasting space and people's time. But if you're absolutely sure you'll need this pattern, then go ahead and use it. It just shouldn't be the default, go-to way of solving this problem.

Re:Do We Worship Complexity? by wwweston on Tuesday October 16, 2018 @03:46PM (#18232558)
Is that the correct hemisphere association? I have a vague idea it'd be the other way around (left has clear proof, right rejects). And what's the book?

I've noticed in myself a tendency towards programming "aesthetics" that just "feel" right. Sometimes that intuition seems to overlap with faculties that help avoid complexity and find elegant solutions... but there's also plenty of times when it's either habit based in a collection of odd assumptions or even apparently arbitrary, and so it's something I've been working to interrogate.

OO approaches specifically have something like three decades of pop-tech discussion as being professional and sophisticated. That's a kind of conditioning that's hard to overcome.

Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @03:52PM (#18232621)
Yes I referred to the correct hemisphere.

The right hemisphere is visuospatial. Also, the right hemisphere 'sees' time/process.

The left hemisphere puts together contextless symbolic model. So I see where your question comes from: the left hemisphere can do the contextless proofs. In this case (and in many cases) the more complex solution "works" and the left brain can "prove" it works.

But a 'simplicity' proof seems better suited to the right hemisphere (I am no expert here, mind you) .... So in my anecdote the "simpler" solution is the one that I would guess the right brain "sees" as simpler: i.e., it is visually much smaller, also much easier to manipulate (over time), etc.

The book is "The Master and His Emissary".

Re:Do We Worship Complexity? by bena on Tuesday October 16, 2018 @03:40PM (#18232506)
Don't over-complicate the reason. ;)

It's quite possible he was trying to test something else and the answer you gave didn't give him a meaningful answer to that question.

So while your solution may have been fine, it's not something you're going to run into in production code. But that hierarchy+command pattern is probably used in a more appropriate situation. And he wanted to see if you could deal with the pattern itself.

Of course, I could be wrong. He could just be enamored with the overly complex solution because it's more "clever".

Interviews are weird.

Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @03:54PM (#18232652)
To be fair, I "passed" the interview. So he got what he was looking for. I argue however that he was preferring complexity over simplicity anyway.
Re:Do We Worship Complexity? by sonnyblarney on Tuesday October 16, 2018 @03:56PM (#18232670)
Hmmmm ...

The point of a 'command pattern' is in effect it's generality, i.e. it implies less coupling in the scenario. The most classic example would be the 'action' that might be passed to a UI button when it's clicked.

So, if the situation calls for command pattern, use it, if not, don't. It's not a matter of 'in your face complexity'.

The notion of doing '3 things with 3 functions' is pedantic: it's obvious. It wouldn't make the basis of a 'question' so it'd be rather pointless to do it.

Maybe there was some confusion as to the point of the question ...

Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @04:01PM (#18232728)
You can slap "command pattern" on anything. But I'm speaking directly in the traditional form that implies a OO type hierarchy.

You don't need that at all.

Here is a "proof" of sorts: http://mishadoff.com/blog/clojure-design-patterns/#episode-1...

Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @04:12PM (#18232827)
Yup. Command pattern is short for "my language doesn't have first-class closures".
Re:Do We Worship Complexity? by sonnyblarney on Tuesday October 16, 2018 @04:17PM (#18232854)
I'm aware.

But either it makes sense to use Command Pattern or not in any given situation and it has nothing to do with complexity really.

I don't see how someone could possibly use a command pattern when simple function would do - that would be beyond pointless. Which makes me believe that there must have been something to the nature of the problem question ...

(Though Lambda's wipe out so many of the simpler use cases of command pattern ...)

Re:Do We Worship Complexity? by Sharlin on Wednesday October 17, 2018 @06:37AM (#18237774)
The whole point was that the GoF Command Pattern is a verbose OO emulation of simply passing around functions. In a language with proper first-class closures the whole pattern mostly dissolves – it’s obvious that if you want to customize behavior, just pass a function. The same holds for a surprising number of GoF patterns.
Re:Do We Worship Complexity? by je42 on Tuesday October 16, 2018 @10:39PM (#18235684)
i think even if you use lambda's - i would still call it a command pattern - if you used lambdas to implement the command class.
Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @06:58AM (#18237864)
It's an issue of perspective at this point. You can call it "The Command Pattern", as if it was something Important, while others will note that it's just passing a closure around (i.e. a trivial programming pattern at the level of "using if/else" or "throwing an exception").
Re:Do We Worship Complexity? by torbjorn on Tuesday October 16, 2018 @06:54PM (#18233991)
Are you by chance reading "The Master and His Emissary"?
Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @08:01PM (#18234372)
Yes that’s the book!
Re:Do We Worship Complexity? by dkarl on Wednesday October 17, 2018 @10:29AM (#18239401)
I had a similar but inverted interview experience years ago. The interviewer asked me how I would implement a system for selling concert tickets online. I suspected, and quickly confirmed, that he wasn't interested in the UI, just the back end systems for supporting the UI. I quickly sketched out a system and described how I would serve up information about the available concerts and tickets and implement the process of buying a ticket. Then I asked the interviewer if he wanted me to go into more detail on any particular component of the system. He was NOT pleased. Then began a long process of "don't you think that's unnecessary?" and "what in the world is this for?" As I explained the need for each part of the system, he started incrementally removing all the requirements implicit in his description of the problem.

First to go was the distinction between accessible and non-accessible seating, partially obscured views, basically any extra info about seats except their identity. Okay, great, that simplifies things.

Likewise any idea of redundancy or failover. "Just let the process restart. You should be able to ensure no failures anyway." Okay, that's kind of opposite to how I've thought in the past, but I'll keep my mouth shut and roll with it.

Just to check, I asked if I should support a concept of buying tickets for assigned seats. Nope, no assigned seating. Okay... so are there different price classes for each event? "No, the tickets are all the same!" Okay. Each event has a single price and a single pool of tickets.

But to sell the ticket the user has to be able to buy it, right? Nope. No payment process. Great! I said. So there's no need to put a hold on a ticket while the user has it in his cart. The interviewer looked like he was going to burst a blood vessel.

Then I said, okay, at minimum I think we need to record who we've sold tickets to so we can send them tickets or in some way ensure that the right people can be let into the concert and people who haven't bought tickets can be turned away. Um... right? "No, don't bother. If they say they bought a ticket, they did." Okay. Moving on.

You wouldn't think the requirements could get much simpler than that, but you would be wrong. It turns out that the system didn't need to know when a concert was happening so it could stop selling tickets at some point. Ticket sales for a concert would go on forever. What should I do if the concert sells out? "Don't worry about that. It won't happen." At this point, I wanted to scream, "Your honor, permission to treat the witness as hostile!"

It also turned out there was no need to track different events or venues. There's only one concert, it has an unlimited capacity, and it never happens.

In the end it emerged that he just wanted a simple TCP service that listened on a port and served a random long integer (a "ticket") to anyone who connected without repeating the same value, unless it got rebooted, in which case it was okay to potentially serve some of the same numbers that it served during its last lifetime, as long as it didn't exhibit any patterns that might enable an attacker to predict upcoming values.

What #%$%ing similarity does that have to selling concert tickets? I mean, in the phrase "sell" "concert" "tickets" "online" three out of the four words were completely irrelevant and misleading. I'm guessing it was just what he was working on that week, and in his system the values were called "tickets" so he threw out "let's sell concert tickets" in an interview hoping I would solve his problem for him. We were out of time, so we didn't have time to discuss it. I didn't get an offer and wouldn't have accepted one.

Re:Do We Worship Complexity? by jcelerier on Tuesday October 16, 2018 @03:39PM (#18232499)
> I remained in Java, but took a functional and dynamic approach yielding a three-line implementation of the engine where each command was simply a function.

okay, so what do I do when I need to dynamically load an additional command by name from an external .jar ? that's like, the most basic thing Java was meant for.

Re:Do We Worship Complexity? by emsy on Tuesday October 16, 2018 @04:04PM (#18232746)
The author describes a YAGNI approach. The interviewer and you seem to want a solution that's already generalized and future proof. YAGNI is always the right approach to take at the beginning of a problem. But that's one of the insights that often come with programming experience and reflection. Unfortunately, seniority is an unreliable indicator for either attribute.
Re:Do We Worship Complexity? by tabtab on Tuesday October 16, 2018 @04:57PM (#18233200)
Perhaps one should ask the interviewer whether they want a YAGNI-style solution or a more general/abstract solution. If you can give both, you may really impress the interviewer.

Asking first to understand the needs and intention of the application should be a first step anyhow. Diving into the deep-end face first and coding is rarely the correct approach (except maybe a startup racing for market-share).

Re:Do We Worship Complexity? by nemo846 on Tuesday October 16, 2018 @08:14PM (#18234430)
> generalized and future proof

Reminds me of a blog post I read,

https://www.sebastiansylvan.com/post/the-perils-of-future-co...

At what point does it become over-engineering?

Re:Do We Worship Complexity? by emsy on Wednesday October 17, 2018 @04:04AM (#18237074)
I think the article captures the problem even better than a similar one that was recently on the frontpage of HN: https://news.ycombinator.com/item?id=17850836

I can't remember a situation where undergeneralized code came back to bite me. I have however seen (and admittedly written some) prematurely generalized code that became messy legacy code.

Re:Do We Worship Complexity? by AlexCoventry on Wednesday October 17, 2018 @12:42AM (#18236177)
You often can't tell up-front. If you never get back more than what you put into an abstraction, it was probably over-engineered.
Re:Do We Worship Complexity? by wvenable on Tuesday October 16, 2018 @04:58PM (#18233214)
You change the implementation. I think the fear of actually changing already existing code leads to the majority of problems with software development.
Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @06:59AM (#18237871)
That's true. We're not writing code in stone tablets; at any point in time you can just dig in and change what you wrote before. It's infinitely easier if that code is simple, because was written straight to the point, instead of being wrapped in large abstractions.
Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @03:55PM (#18232660)
This should be in requirements. It apparently wasn't in the ones OP was given.

Also, the answer should not be "large dependency injection framework".

Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @03:59PM (#18232712)
Also I would say the answer is better left independent of the runtime environment.

Yes you can load Java classes into the JVM. But why not have a serialization format that is independent of the vagaries and specifities of the JVM?

Still, even if you lean on the JVM, that changes nothing about this particular problem. You don't need command pattern. You don't need a type hierarchy.

Re:Do We Worship Complexity? by ensiferum on Wednesday October 17, 2018 @05:15AM (#18237424)
You forgot some Factories and Factory Factories ;-)
Re:Do We Worship Complexity? by choot on Wednesday October 17, 2018 @02:41AM (#18236683)
One of my friend who was not so strong in programming was never short of any jobs that were available to him.

He told me he took a ukulele to the interview. Each time when interviewer asked him a question, he would play ukulele untill he found the answer.

He got the job even when his interviews went wrong.

His job offer arrived months after an interview because he was the only guy who the interviewer could remember.

Often you hire for a spot in a company, after sometime that person leaves and management asks you to name any other person and when you try to recall last interviewee batch, the ukulele guy is only one the interviewer is able to remember.

Re:Do We Worship Complexity? by heavenlyblue on Thursday October 18, 2018 @07:08AM (#18247336)
I've spent a fair amount of time interviewing in London a few years ago. I would literally go to any company that send me an e-mail or contacted me via recruiters.

I didn't have enough money to buy new clothes at the time, so instead I've picked the shoes with the largest amount of holes and would never do any grooming before hands. I would come stoned to the interview. So the gist: instead of trying to appear on the lower mid end, I went all the way through to the lowest low end.

Thinking retrospectively I think that's pretty much the reason why I barely received any negative responses unless I had miserably failed at the technical side of things (e.g. I didn't know how to use generators in Python at the time and the whole list of questions would be about them and their syntax).

I have to admit that in this scenario I was quite good at the technical side of things. But the general philosophy was that if I don't try to appear too good - people will assume that I was better. Put humility and definite sprinkle of character on top of that.

Re:Do We Worship Complexity? by Unknoob on Wednesday October 17, 2018 @09:01AM (#18238507)
Your friend is a mad genius. That's so brilliantly absurd but aligns so well with my experience in interviews that I can't dispute it.
Re:Do We Worship Complexity? by mercer on Wednesday October 17, 2018 @11:22AM (#18239950)
That's awesome, and there must be more stories you can tell about this guy. If so, do tell!
Re:Do We Worship Complexity? by twtw on Tuesday October 16, 2018 @03:36PM (#18232471)
> What we ended up with was O(n) _classes_ (where n = the number of computation operations).

Speaking of complexity worship...

Big O notation seems totally irrelevant here, unless the question involved scaling to arbitrary number of types of operations. Why not just say "we ended up with a class for every operation?"

Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @03:53PM (#18232638)
Big-O applied to code artifacts-to-manage-over-time is very relevant here. If not the common usage of Big-O, which is in terms of algorithmic steps. But still highly relevant.

A system that has O(n) code artifacts for n "business requirements" (e.g.) is far more costly than one that has O(1), say.

Re:Do We Worship Complexity? by twtw on Tuesday October 16, 2018 @04:04PM (#18232745)
> A system that has O(n) code artifacts for n "business requirements" (e.g.) is far more costly than one that has O(1), say.

I understand what you are saying, I'm just objecting to how you are saying it. Unless you are interested in the limiting behavior as the number of "business requirements" goes to infinity, big O is the wrong tool. An O(1) approach that requires a foundation of 1M LOC is probably worse than an O(n) approach that requires 100 LOC per business requirement.

Sorry to obsess over the details here - I just think big O is overused in general and saw an opportunity to make a bad pun on complexity worship.

Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @04:08PM (#18232787)
"Big O" has evolved beyond its original intended usage. In informal and semi-formal setting, it just means "roughy proportional to". It's showcasing the first-order shape of growth, not just inviting to do analysis in the limit.

I.e. it's coder slang now.

Re:Do We Worship Complexity? by twtw on Tuesday October 16, 2018 @04:14PM (#18232845)
I understand and I'm well aware that it has become "coder slang" - that's my point. On a thread about complexity worship, the first comment I saw was an unnecessary use of jargon that complicates things without new meaning. That, in my mind, results from a culture of complexity worship.

> Never use a foreign phrase, a scientific word, or a jargon word if you can think of an everyday English equivalent. - orwell

Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @04:24PM (#18232919)
I think the slang version of "big O" is so common that there's no chance of misunderstanding. If anything, it serves as more of a shibboleth - if you don't get it, you're not part of the group the story is intended for.

RE that Orwell quote, I have mixed feelings. I agree with it in so far as it means "pick easiest words for your audience at the precision level you need". But in general, words are not equivalents, even if they're listed as synonyms - each word has its own specific connotation. Say, "car" and "automobile". Technically, they refer to the same thing, but they feel different. That subtle emotional difference may not be important in formal setting, but it's an extra dimension of communication in informal cases (like e.g. this comment thread).

Re:Do We Worship Complexity? by tomnipotent on Tuesday October 16, 2018 @04:56PM (#18233186)
> it serves as more of a shibboleth

Like the word "orthogonal" - always know I'm talking to a techie when that one rears its head.

Re:Do We Worship Complexity? by derefr on Tuesday October 16, 2018 @07:18PM (#18234138)
I'd say "perpendicular" but that's somehow even more niche, despite being a word everyone learns in high-school geometry.

I guess there just comes a point where you've solved so many optimization problems that it's hard to not think of a bunch of solutions with different attributes as being embedded in an N-dimensional space?

Re:Do We Worship Complexity? by humanrebar on Tuesday October 16, 2018 @09:19PM (#18234783)
That is an orthogonal concern.

Versus

That is an entirely different concern.

The second is more approachable but requires an adverb to have the same meaning, and adverbs are also discouraged. I think orthogonal is fine.

Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @02:36AM (#18236651)
Also it does not communicate the precise meaning. "Orthogonal" fits when concerns may be related, but they're independent from one another, so you can discuss them separately.
Re:Do We Worship Complexity? by humanrebar on Wednesday October 17, 2018 @07:09AM (#18237917)
Well, regardless of the adverb, you would need a modifier. Avoiding these technical considerations was part of Orwell's point I think.

So I guess we could nitpick over "entirely" and "basically" just as easily. Maybe he should have just singled out the engineers and other pedants.

Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @02:34AM (#18236645)
Perpendicular is a weird word. And also harder to say, at least for foreigners.

I tend to use "mutually independent" instead of "orthogonal" when talking with non-tech people.

Re:Do We Worship Complexity? by theoh on Tuesday October 16, 2018 @04:57PM (#18233193)
That (Big O as slang) sounds like a horrible situation.

It feels analogous to the widespread use of "exponentially" to mean "a lot" or "quickly" which is a really bad, silly thing. The difference is that few physicists and mathematicians misuse "exponentially" in casual conversation, whereas you are claiming that software people deliberately misuse "Big O". I'm not sure I believe you but either way this seems regrettable.

Re:Do We Worship Complexity? by derefr on Tuesday October 16, 2018 @07:21PM (#18234150)
"Exponentially" isn't all that bad; what people actually mean when they say it is usually "superlinearly", but there's no practical difference between the two when talking about e.g. scaling problems.
Re:Do We Worship Complexity? by theoh on Wednesday October 17, 2018 @07:14PM (#18244640)
YMMV, but, outside of technical contexts, the phrase "increased exponentially" is often used by people who don't even know the difference between linear, geometric and exponential growth. In many cases they don't even understand that the word "exponential" refers to a rate of growth, if they've heard of the concept.

Take the context of a high-profile art magazine, Frieze. (I googled "frieze increased exponentially"). This is shooting fish in a barrel—but the most egregious example in the first page of hits is this one:

"Seppie in nero’ – squid in its own ink – is my favourite Venetian delicacy. Although customarily served with polenta, I prefer it on thick spaghetti since pasta exponentially increases the naturally squirmy quality of the creatures’ tentacles, creating a Medusa-like mound of inchoate, salty matter."

So you've got an art critic writing slightly pretentiously about food, and he throws in an "exponentially" which has nothing to do with a rate.

This and similar usages of "exponentially" are extremely widespread now. People talk about exponential increases without any mental model of the rate of growth as a function of time at all—just the woolly idea that something is growing fast.

"The term "exponentially" is often used to convey that a value has taken a big jump in short period of time, but to say that a value has changed exponentially does not necessarily mean that it has grown very much in that particular moment, but rather that the rate at which it grows is described by an exponential function."

https://books.google.ie/books?id=aVovDwAAQBAJ&pg=PA36&lpg=PA...

The battle is lost on this word, as it is with "literally". To the man on the street "exponentially" really just means "a huge amount" now.

Re:Do We Worship Complexity? by apetresc on Tuesday October 16, 2018 @09:38PM (#18234975)
There is absolutely a very practical difference.
Re:Do We Worship Complexity? by SilasX on Tuesday October 16, 2018 @05:23PM (#18233407)
It's unfair to dismiss that as sloppy slang; that's exactly what Big O notation exists for: to concisely express such concepts as "the number of classes increases linearly with the number of business requirements", as differentiated from "the number of classes is independent of business requirements" or "the number of classes increases with the square of business requirements".

The Orwell quote is valid at the heuristic level, but when

a) there's an installed base of people who know the jargon, and

b) the everyday English equivalent takes a lot more words to say the same thing, and

c) something coherent is meant by the jargon that could be so translated if necessary,

then that's exactly when you should use the jargon.

Give me "a^2 + b^2 = c^2" over "the sum of the squares of the lengths of a and b is equal to the square of the length of c".

Re:Do We Worship Complexity? by dfox on Tuesday October 16, 2018 @06:18PM (#18233782)
The point there is that Big-O notation also implies that the growth has given shape above some large-ish n. And metrics like "number of operators" or "number of bussiness requirements" rarely are so large that this makes sense.

And in fact approaches to system design that try to make the code size independent of number of bussiness requirements and their possible changes in future lead to exactly the kind of "complexity worship" discussed in TFA (various ad-hoc turing complete VMs that interpret code represented as rows in bunch of relational tables and what not).

Re:Do We Worship Complexity? by derefr on Tuesday October 16, 2018 @07:22PM (#18234163)
People who write "x is O(n)" usually actually mean to write "x ∝ n", but 1. ∝ is hard to type on a keyboard, and 2. even fewer people (including CS majors!) know what ∝ means, than know what O(n) means well-enough to allow them to figure out its non-jargon usage from context.
Re:Do We Worship Complexity? by SilasX on Tuesday October 16, 2018 @06:28PM (#18233845)
So you think that the imprecision of saying "O(n)" when the "large-ish" behavior is not relevant is worse than the verbosity of saying "scales directly/with the square of/not-at-all with [relevant constraint]"?

FWIW, Big-O itself, even in technical contexts, gets imprecise with e.g. calling hashtables O(1), which is not possible, even under the idealized computer model (instant memory access, etc).

Is there a shorter way of saying "scales proportionally with n" that you would suggest the tech community prefer because of its greater precision?

Re:Do We Worship Complexity? by twtw on Tuesday October 16, 2018 @06:39PM (#18233918)
But you can't just say "O(n)," you have to specify what n is - at which point it is no longer any more terse than the English alternative.

> What we ended up with was O(n) _classes_ (where n = the number of computation operations).

vs.

> what we ended up with was several classes for each operation.

The second is shorter, and says no more than what is relevant.

Re:Do We Worship Complexity? by SilasX on Tuesday October 16, 2018 @06:47PM (#18233955)
That's like saying there's no point in using pronouns, since you have to say the antecedent anyway.

It would be wrong in both cases because the context can make clear what a variable or pronoun refers to. If the problem context makes clear what the binding constraint is and you just need to talk about scaling behavior, then it is indeed shorter to say "O(1) rather than O(n)" vs "doesn't depend on the operations rather than being directly proportional".

>>what we ended up with was several classes for each operation.

>The second is shorter, and says no more than what is relevant.

It says less: the O notation is used to indicate that as you add more operations, you will need to add more classes, rather than only needing to add classes when there is logic the operations don't yet implement.

Re:Do We Worship Complexity? by TeMPOraL on Wednesday October 17, 2018 @02:40AM (#18236680)
Math doesn't define how big a "large-ish n" is. In casual usage, it may as well be n > 3, if the context suggests so.
Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @04:19PM (#18232869)
Got you. Thanks for the clarification.

Speaking of brain hemispheres, the left hemisphere doesn't get jokes like the one you made. The right hemisphere handles jokes, metaphors, etc.

Looks like I'm a left-hemisphere-dominant pot calling the kettle black...

Re:Do We Worship Complexity? by taeric on Tuesday October 16, 2018 @04:06PM (#18232767)
Tough to say, really. If your O(1) is just a constant 2000 code artifacts, but your O(n) is 5 per business requirement, it is amusing to consider that 400 business requirements might be better spread over more systems.

My view is that, even in algorithm analysis, Big-O has been overused. Too many people will point out crap like "That is 4 items per entity!" When I point out we only have 50 entities and that 200 is an easily handled number, I just get evil glares.

Re:Do We Worship Complexity? by MaxBarraclough on Tuesday October 16, 2018 @06:09PM (#18233735)
Somewhat related: I've always enjoyed that the matrix multiplication algorithms with the best theoretical time complexity, are completely impractical for use on real-world matrix multiplication problems.

Wikipedia describes why people use the Strassen algorithm in real-world implementations, despite its inferior asymptotic time complexity [0] :

> unlike the Strassen algorithm, it is not used in practice because it only provides an advantage for matrices so large that they cannot be processed by modern hardware.

[0] https://en.wikipedia.org/wiki/Coppersmith%E2%80%93Winograd_a...

Re:Do We Worship Complexity? by taeric on Tuesday October 16, 2018 @06:33PM (#18233884)
Had a fun discussion on that in this forum not long ago. https://news.ycombinator.com/item?id=17891360

Basically, I was hoping some of the optimized methods were competitive nowdays. Spoiler, still not practical. :(

Re:Do We Worship Complexity? by robotresearcher on Tuesday October 16, 2018 @03:50PM (#18232600)
Perhaps because they had a number of classes per operation, but that number was not important. Statements like this is what big-O is for.
Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @03:53PM (#18232637)
The use of that notation implies the number of classes was proportional to number of operations (+/- constant factor), not that it was equal to number of operations. O(n) works both if there's 1 class per operation, and if there's 5 classes per operation.
Re:Do We Worship Complexity? by twtw on Tuesday October 16, 2018 @04:07PM (#18232776)
Ok. O(n) also through a away a lot of useful information. It seems simpler in every sense to just say "several classes for every operation."
Re:Do We Worship Complexity? by TeMPOraL on Tuesday October 16, 2018 @04:09PM (#18232807)
Again, you're absolutely right wrt. proper, formal usage. But big-O is also a part of programmer slang now. The OP used it in this form.

(It's also faster to write, or even say out loud, "O(n)", than it is to write/say "proportional to number of").

Re:Do We Worship Complexity? by derefr on Tuesday October 16, 2018 @07:29PM (#18234203)
Frequently when writing O(n), the writer is making a comparison. e.g. "The GC was O(n log n) wrt memory, but now it's just O(log n)." That expands to a lot more text.

Or, to put that another way: there is a constant cost in human parsing complexity when writing "O(n)", but the number of parse-nodes in the text when using "O(n)" only increases as O(n), whereas without "O(n)", it increases with O(n^1.3). ;)

Re:Do We Worship Complexity? by gaius on Wednesday October 17, 2018 @03:29AM (#18236911)
I'm currently reading a book on brain hemispheres. Apparently experiments have shown that the left hemisphere will completely and blatantly reject obvious evidence even if the right hemisphere 'has a clear counter proof'. Sometimes I think our industry suffers from an abundance of left hemispheric dominance.

I have two monitors on my desk. If I am stuck on anything I drag my emacs from one side to the other and look at the same code there. It works pretty well!

Re:Do We Worship Complexity? by crimsonalucard on Tuesday October 16, 2018 @04:11PM (#18232823)
A 3 line simple solution is obviously better than the original solution. He knows this, Anybody knows this.

What happened was that you showed off how you're better than him at finding a better solution. He's probably a top dog looking for underlings. That's the reality of this world.

Re:Do We Worship Complexity? by wellpast on Tuesday October 16, 2018 @04:21PM (#18232891)
I can appreciate your somewhat cynical assessment, but I also think we're still squarely in the right/left brain hemisphere: empirically speaking, the left hemisphere has quite the ego. We're both saying a very similar thing.
Re:Do We Worship Complexity? by Annatar on Wednesday October 17, 2018 @04:27AM (#18237203)
Your situation was a perfect example of why I so violently, vehemently oppose object oriented programming and why I fight the people trying to promote it at every turn and opportunity (often in person and it can get extremely ugly -- and that is good so). Object oriented code is a nightmare to debug -- I've wasted countless years stepping through the debugger trying to follow the complex state machines I was debugging. Data hiding, one of the main selling points of this programming technique, makes it even worse with regard to building a mental model of the state of the machine. Resolving what the data is in order to understand where the problem resides ends up being extremely tedious and painful.

It's like they are trying to prove that they are really, really smart without anyone asking them to prove it. But here is my challenge: anyone can overcomplicate anything, but if one is really smart, make a complex thing simple.

Re:Do We Worship Complexity? by zmmmmm on Wednesday October 17, 2018 @01:42AM (#18236394)
Very likely the interviewer was posing to you the exact design that they have already implemented, which they did for good reasons, and very likely your trivial solution would not scale up to the complexity of their needs. So you might have proved you could simplify the problem at hand but failed to demonstrate that you would know how to deal with more complex requirements of a larger system. For example if the commands themselves are quite complicated and have their own relationships and state then how would that be modeled? Eg. how do commands with shared behavior share it? Of course it can all be done in a functional style if you are determined enough, but it could be somewhat awkward and sounds like it wouldn't fit at all with their style of programming. I assume you probably didn't want the job in the end but if you did I would say you didn't do yourself any favors.