News for hackers. Stuff that matters.
Re:Propellerhead Software - Application Programmer Test by ekidd on Friday May 20, 2011 @08:13AM (#2567315)
This test would be much more useful if they put all the errors into a broken string class, and asked you to find the bugs. Glancing at a broken class and seeing all the standard C++ bugs "pop out" is a useful skill for a C++ programmer. But instead of defining an actual, useful class with a bunch of bugs, they use horrendously nonsensical and broken code fragments. For example, I can't even figure out what the GrowArray function is supposed to do.

It's sort of like memorizing a chess board: A chess expert can memorize an actual game-in-progress quickly. But if you just place a bunch of random pieces around a chess board, a chess expert can't memorize it any better than anybody else.

The later examples with memchr are less painful, because memchr actually has well-specified behavior. But even there, the code (most likely) dereferences a pointer one past the end of an array, which is (1) invalid C++ and (2) not on their list of possible bugs. Granted, memchr is an internal compiler function, so it's allowed to rely on undefined behavior. But if this is a multithreaded system, that code potentially corrupts the heap's bookkeeping data structures while memchr is running.

Just looking at the first part of this test is like listening to somebody drag their fingernails across a blackboard. I'm sure that they're a lovely company, but ugh. To paraphrase Wolfgang Pauli, that code's so bad it's not even buggy—it's just semi-coherent nonsense.

Another interesting aspect of this test: Both the code examples and the bugs are pre-STL, pre-boost, and pre-TR1. For example, there's no question asking why you can't put a std::auto_ptr into an STL collection. And a huge fraction of the bugs in the test could be avoided by making rudimentary use of modern C++ APIs. Granted, some companies have legitimate reasons to avoid std::vector and std::tr1::shared_ptr. But avoiding those classes makes C++ vastly more error prone, which is usually a poor tradeoff in the modern world.

Re:Propellerhead Software - Application Programmer Test by davisp on Friday May 20, 2011 @11:15AM (#2567980)
Its been awhile since I C++'ed heavily, but I think this isn't quite right:

"But even there, the code (most likely) dereferences a pointer one past the end of an array, which is (1) invalid C++ and (2) not on their list of possible bugs."

There's a weirdo thing with array allocation in C++ at least which makes it valid to use arrays with all of the std:: algorithms that take a start and end parameter. Unfortunately I can't remember it well enough to find a good reference but as I recall it was more than just a pointer comparison as the std:: algorithms wanted to be able to dereference it occasionally.

Then again its been about four years since I've touched C++ so some of these details have leaked out and gotten mixed up.

Re:Propellerhead Software - Application Programmer Test by ekidd on Friday May 20, 2011 @11:41AM (#2568093)
There's a weirdo thing with array allocation in C++ at least which makes it valid to use arrays with all of the std:: algorithms that take a start and end parameter.

The actual rule is pretty subtle, and I don't have a copy of the C++ standard in front of me. But let's see if I can remember the precise rule.

Given an array arr with length len, you're allowed to create the pointer arr+len. This is the end pointer you see everywhere in modern C++ APIs. However, if you try to deference arr+len, or try to create the pointer arr+len+1, the result is undefined. Typically, the former might point to a locked page, and the latter might overflow the pointer size. But if your compiler applies very aggressive optimizations, even stranger stuff can happen when you rely on undefined behavior:

http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

In the memchr example, the code potentially writes to arr+len, which has a decent chance of temporarily corrupting the implementation-dependent internal headers of the next heap block. In the presence of threads, this would very occasionally corrupt the internal state of new and delete, and debugging it would be a stupendously painful experience.

Re:Propellerhead Software - Application Programmer Test by cageface on Friday May 20, 2011 @07:53AM (#2567271)
These guys are behind what is easily the most robust and reliable audio app on the market (Reason). Whatever their code practices are internally, they're doing something right.
Re:Propellerhead Software - Application Programmer Test by paulnelligan on Friday May 20, 2011 @08:12AM (#2567314)
True, Reason has been around a long time and has always been an awesome piece of kit - apparently it got prodigy's Liam Howlett back into writing after years of hiatus, we can thank propellerheads for their part in 'always outnumbered, never outgunned' ...
Re:Propellerhead Software - Application Programmer Test by eropple on Friday May 20, 2011 @10:47AM (#2567854)
Easily? Meeeeh. I really, really like Reason, but for me it lives as a bag on the side of Ableton (Record doesn't cut it) and I've never had an issue with Ableton either.
Re:Propellerhead Software - Application Programmer Test by cageface on Friday May 20, 2011 @08:00PM (#2569839)
I love Live but Ableton themselves admitted that Live 8 wasn't as stable as it should have been and stopped production on Live 9 to iron out the many, many crash bugs in Live 8.

Reason may not be the most full-featured audio app but it's certainly the most stable.

Re:Propellerhead Software - Application Programmer Test by eropple on Friday May 20, 2011 @11:23PM (#2570166)
Really? Huh, I've never had a stability problem with it (even with my own home-rolled VSTs). The more you know...
Re:Propellerhead Software - Application Programmer Test by Deestan on Friday May 20, 2011 @09:56AM (#2567640)
Q1 and Q2 are horrible, but it picks up a bit after that.

Q1: I see a problem on all lines 1 through 6: The class doesn't do anything. There is also no documentation as to what it is intended to do. Here it becomes, as leon_ also comments, a game of "guess what the test writer wanted you to answer".

Q2: "It will work if..." is a meaningless phrase, because I am not told what the code is supposed to do.

Q12: "How many books have you read this year?" It is immediately obvious that "5 or more" is the answer that will give me the highest ranking from their application sorting robot. Everyone is going to answer "5 or more". The honest ones will then flip through the needed amount of books before the interview. The question is as meaningless as "are you a good developer or a bad developer?"

Re:Propellerhead Software - Application Programmer Test by motters on Friday May 20, 2011 @10:09AM (#2567692)
The main problem with these trick programs is that as a software engineer of n years I don't debug programs by staring at them for a long time or by memorising a lot of tricks. I usually do it by running them and checking their output, or by catching runtime errors in a debugger. Writing software is not a one way process, it's a skill or an interactive process with the machine.
Re:Propellerhead Software - Application Programmer Test by Vlasta on Friday May 20, 2011 @08:41AM (#2567388)
In my book their value "1) Good interface: safe to use - hard to misuse" is the same as their non-value "3) Grand design for the future".
Re:Propellerhead Software - Application Programmer Test by scottdw2 on Friday May 20, 2011 @09:29AM (#2567540)
I think is may not be testing the right things.

For example, what if the test was in Visual Basic, and instead of asking questions about non virtual destructors in virtual classes, it was asking questions about uses of various database Apis. " Oops, you used the wrong enum value in that function call."

Would that test yield top programmers, or just people who knew a database API very well?

Also, it doesn't demonstrate the ability to solve problems by writing code. At best it tests C++ code review skills.

Re:Propellerhead Software - Application Programmer Test by tomjen3 on Friday May 20, 2011 @06:48AM (#2567143)
Unfortunately what half of those questions tests are pointless - don't test C++ minutia (especially not in 2011), test whether the candidate can code.
Re:Propellerhead Software - Application Programmer Test by bandushrew on Friday May 20, 2011 @07:35AM (#2567229)
They weren't testing minutia, they were testing depth of understanding and experience. I thought they did a decent job, too.

I would have preferred to be able to get a result without actually applying for the job though...

Re:Propellerhead Software - Application Programmer Test by jt2190 on Friday May 20, 2011 @08:13AM (#2567319)
Having never programmed a line of C++ in my life, I started to guess at the first question. After 30 seconds or so, I thought: "These programs are short enough to just type in, compile and run. Screw using my 'experience' to answer these." I guess once I work through these they'll know that I can compile and run a program in C++, and perhaps that's all they're trying to learn here.
Re:Propellerhead Software - Application Programmer Test by ctz on Friday May 20, 2011 @08:08AM (#2567303)
Q10 is missing the correct answer: "someone inexplicably decided to implement part of the C standard library in C++".

Not to mention the function's name will be mangled, making it unusable for C users.

Re:Propellerhead Software - Application Programmer Test by theDoug on Friday May 20, 2011 @10:34AM (#2567796)
Facebook has a (somewhat) similar process: http://www.facebook.com/careers/puzzles.php
Re:Propellerhead Software - Application Programmer Test by roel_v on Friday May 20, 2011 @07:07AM (#2567178)
This test sucks - I just spend 15 minutes and I don't even get a result!
Re:Propellerhead Software - Application Programmer Test by leon_ on Friday May 20, 2011 @08:20AM (#2567337)
Warning, some spoilers ...

I don't like this kind of tests. I always stumble upon questions where I wonder if it's a trick question or just a typo/mistake by the test creator. Example:

Function D will loop for ever if size_t is an unsigned type. Function E won't even compile because of the comma and the missing * in the first line. Function C is an edge case. The external effect is the same only in a single threaded environment. If there's parallelism or interrupts the external effect might not be the same for certain conditions.

So what do I do? Do I check the box or not? If I do, it might fire back because "I didn't catch those obvious errors". If I don't then my application might fall through the grid because "too many wrong checkmarks". Maybe the recruiter is just a mindless HR bot and doesn't understand any of these questions. So arguing would be pointless.

If you want to determine my skills, give me something to do. Say give me 6 hours to implement a server or something (not too big, not too small for 6 hours), then shuffle through my code and see if its quality pleases you. Being bombarded with off-by-one trick questions just annoys me.