Re:Codility - Automated tests of programming skills
by defdac on Friday January 08, 2010 @06:06AM
(#1039271)
Very good idea, but I don't program that way. I use a competent IDE to tell me syntax errors and obvious programming mistakes while I write. If I can't debug my algorithm with trial & error I write a test first and modify my algorithm until it complies with the test.
To think that a programmer will always come up with the exact and most optimal solution to any given problem is a bit naïve. In my 10 years as a programmer I have never seen one wonder-kid that is able to do that. The real wonder kid assets in a team also tend to be those that are more creative than syntax-anal compared to those that can write syntax-error-free-code with pen and pencil and remember large parts of a phonebook...
Re:Codility - Automated tests of programming skills
by ramchip on Friday January 08, 2010 @06:41AM
(#1039306)
To think that a programmer will always come up with the exact and most optimal solution to any given problem is a bit naïve
I have no idea where you saw this being implied. The software highlights syntax and lets you check your code as many times as desired, as well as giving hints if your result isn't correct and providing compiler output. Nothing stops you from googling something. It's not like the software will destroy your file and egg your house because your algorithm is o(n2) instead of o(n); that's up to whoever is evaluating the results.
The fact that it has a polish and chinese version should also give you a little hint about the intended audience.
Re:Codility - Automated tests of programming skills
by marcin on Friday January 08, 2010 @06:19AM
(#1039282)
Well, that's one more reason you should like Codility - you can run test compilations (the VERIFY button) which will show you syntax errors. And in bottom left corner you can apply your test data sets. In other words - you work like in your every day environment. Check it out and let me know what you think.
Re:Codility - Automated tests of programming skills
by defdac on Friday January 08, 2010 @06:54AM
(#1039324)
I did, and now I see comments stating that you should be able to run tests on your code while developing it. That's awesome and what I was missing.
Next step intellisense and on-the-fly syntax checking. I hope no programmers click the "compile"-button nowadays to check if their code is correct...
Re:Codility - Automated tests of programming skills
by JoachimSchipper on Friday January 08, 2010 @09:27AM
(#1039552)
This is quite nice, even if the exercise isn't all that difficult.
It would be nice if the boundary conditions were better specified (people seem to have been bitten by overflowing integers). This could be inferred if you told people that you're using a 32-bit machine (the sum of elements is at most SIZE_MAX * INT_MAX; if int and size_t are 32-bit, SIZE_MAX * INT_MAX < INT64_MAX.)
I was also somewhat surprised to see that my code compiles without #include <stdint.h>. I'm not sure this is correct (it could be solved by putting the code written by the candidate before the test code).
Finally, I assume you have proper sandboxing set up?
Re:Codility - Automated tests of programming skills
by cf2000 on Friday January 08, 2010 @07:02PM
(#1040754)
I think this is cute but I would think hard about the kinds of questions you develop for this. Coming from the perspective of someone working for a big company with a ton of software people that is not a tech company, I could use something like this in general. But, the question you asked is the kind of thing I ask CS undergrads (almost exactly, in fact) because it allows for clever solutions without actually requiring someone to be a particularly good programmer. I can work through it with them and if the basic solution is easy, prompt them to give me something more clever, so I know how they think, which is good when you're looking at an inexperienced developer. However, for an experienced developer that I want to hire to churn out Java code, the difference to me between a correct but simple O(n^2) solution vs a clever O(n) is not terribly meaningful for determining whether they can actually do what I want them to do. I want something that requires people to write more than a handful of lines and demonstrates an ability to keep track of a few more variables correctly, and honestly I don't care all that much about their algorithmic complexity (although I might care a great deal about their space usage, usage of external resources, etc).
So, if this is the audience you are looking to attract, consider thinking about this as a screening tool for people outside of the startup world that actually need "knows how to really do shit in language X" more than "knows how to write clever algorithm,"
Re:Codility - Automated tests of programming skills
by jakacki on Monday January 11, 2010 @04:36PM
(#1046088)
I don't think Codility checks whether one knows how to write clever algorithm. Look at tasks from ACM Collegiate Programming Competitions, they require clever algorithms. Codility deliberately focuses on very fundamental tasks. Also it does not pretend to be a final sign-off, it merely tells you which candidates to reject due to lack of rudimentary programming skills. It's not that surgeons make injections that often in their daily practice, yet I would not trust a surgeon who doesn't know how to make an injection.
Re:Codility - Automated tests of programming skills
by charger on Thursday February 11, 2010 @11:55PM
(#1119811)
I like this comment.
A typical surgeon earns a bachelors degree, then attends medical school, before finally spending three to six years in a residency program (in the United States). This is before becoming a practicing surgeon.
Unfortunately, there is nothing similar (in the United States) that is related to software engineering -- though many here would like to see something similar to an engineering license for technology professionals; there is nothing currently on the horizon (that I know of).
In some ways, I think that Codility can help with this state of affairs. This is particularly interesting when we consider comparing metrics for education and experience; and perhaps 'real-world' success.
The Codility statistical comparison, taking these factors into account, would be very interesting indeed. And could possibly result in data that furthers the state of the art in general.
Re:Codility - Automated tests of programming skills
by ununoctio on Sunday March 21, 2010 @05:00AM
(#1207723)
{In Pascal Language+Assembly}
{$ASMMODE INTEL}
function equi (A : Array of longint; n : longint ) : longint;
var c:Longint;
label noOverflow1;
label noOverflow2;
label ciclo;
label fine;
label Over;
label tot;
Begin
Asm
DEC n
JS over
XOR ECX, ECX {Somma1}
XOR EDI, EDI {Somma2}
XOR EAX, EAX
MOV c, EDI
MOV ESI, n
tot:
MOV EDX, A
MOV EDX, [EDX+ESI*4]
PUSH EDX
ADD ECX, EDX
JNO nooverflow1
ADD c, ECX
nooverflow1:
DEC ESI
JNS tot;
SUB ECX, c
SUB EDI, c
ciclo:
POP EDX
SUB ECX, EDX
CMP ECX, EDI
JE fine
ADD EDI, EDX
JNO nooverflow2
DEC EDI
nooverflow2:
CMP EAX, n
JA over
INC EAX
JMP ciclo
over:
MOV EAX, -1
fine:
end;
End;
Re:Codility - Automated tests of programming skills
by jakacki on Friday January 08, 2010 @06:36PM
(#1040712)
Generally we don't want to penalize people too badly for assuming particular (but realistic) type ranges, nevertheless we think that perfect solution should be as portable as possible across different implementations of the language (e.g. one should not assume that C's int is 32 bits wide, since implementation with 16-bit int can still be Standard compliant)
Re:Codility - Automated tests of programming skills
by patio11 on Friday January 08, 2010 @08:15AM
(#1039432)
That is an amazing piece of software. I hope you make a mint.
I'd give people access to their language's output facility for playing around to the solution, but that is a minor nitpick. (And I suppose I could open an irb instance on my machine for exploration and then give you code when I'm good and ready.)
My Ruby solution, in case we've got anybody who is struggling: http://pastie.org/771438 This would be my first cut if you asked me to whiteboard this.
The site's very slick. My only feedback is the histogram on the results page could be more prominent - it's cool, useful information but I ended up inspecting the html elements to work out what it was.
Also the results page seemed to keep refreshing for me while waiting for results (Safari 3). Surely that should've been AJAX'd?
Re:Codility - Automated tests of programming skills
by moconnor on Tuesday January 12, 2010 @03:08AM
(#1047099)
+1! Thanks for the reminder. I'm fairly new to python and couldn't remember the keyword. I guess this is a reasonable argument for the time deadline - to get it done quickly I didn't look it up, instead doing the simplest thing I knew would work and thus exposing the pitiful state of my python experience :-)
Re:Codility - Automated tests of programming skills
by almost on Friday January 08, 2010 @11:16AM
(#1039798)
enumerate is very useful, here's mine:
def equi ( A ):
right = sum(A)
left = 0
for i,x in enumerate(A):
right -= x
if right == left:
return i
left += x
return -1
What's interesting is that all the versions I've seen so far are pretty much identical apart from very minor things. This would seem to me to be obviously the correct way to do it but I'm interested if it's just a way. Did anyone solve this in a different way of equal or greater elegance?
Re:Codility - Automated tests of programming skills
by anc2020 on Friday January 08, 2010 @03:32PM
(#1040355)
Mine kept one cumulative list adding left to right and another summing right to left. This is conventionally slower, but conceivably faster if you do it in parallel, leaving the final loop to be something like:
for i in xrange(len(A)):
if right[i] == left[i]:
return i
The final loop here is slightly tighter, so as long as your parallelism didn't add big overheads, this will be faster.
Re:Codility - Automated tests of programming skills
by samstokes on Friday January 08, 2010 @04:19PM
(#1040459)
I like how the performance evaluation seems to be fairly realistic - I used Ruby and my algorithm could have been more efficient (O(n) space, and O(n) time but with more traversals than necessary), but I scored 100 because I met all the stated constraints. (http://codility.com/demo/results/?id=demo8XKE77-H8G if anyone wants to refer to and/or mock my code.) It seems to me "ability to solve problems well enough under time pressure" is exactly what you want a simple hands-off test like this to screen for.
In face-to-face interviews you'd probably follow up by adding constraints: "your algorithm uses O(n) space in addition to the input, can you do it in constant space?". It would be cool if Codility supported that kind of workflow - following up the submitted solution with refinements on the original question. (Relatedly - do you support the workflow where you get a second try if you didn't get 100, maybe because the evalution revealed a constraint that wasn't clearly stated in the problem outline?)
Re:Codility - Automated tests of programming skills
by jakacki on Monday January 11, 2010 @04:41PM
(#1046097)
With paid account it is possible to reopen a task after one solution has been submitted, so the recruiter can "give another chance" to the candidate (although we generally suggest not to), which supports the workflow you describe.
Re:Codility - Automated tests of programming skills
by Erwin on Friday January 08, 2010 @06:49AM
(#1039317)
Is the code editor they're using something publicly available? I'm looking around for a light in-browser code editor with programmer-oriented features (e.g. syntax highlighting, this also seems to auto-indent Python code to some degree).
Re:Codility - Automated tests of programming skills
by ideamonk on Friday January 08, 2010 @07:32AM
(#1039360)
Even I noticed that I had seen something ditto somewhere... Hmm so Codility's editor is copy of the one SPOJ uses, so which one do they both use? Can I too get it :)
Re:Codility - Automated tests of programming skills
by tudorachim on Friday January 08, 2010 @10:16AM
(#1039671)
If I have an account and am taking a test that I've created, you should provide a link back to the dashboard after I've submitted so I can check how I did. Currently, I'm taken to the the "enter a password" screen and have to manually go back to the homepage and then to the dashboard.
Re:Codility - Automated tests of programming skills
by Estragon on Friday January 08, 2010 @09:31AM
(#1039559)
It gave me a test concerning finding the "equilibrium index" of a sequence. It said to assume that the sequence is very long. (The question is repeated here: http://forums.sun.com/thread.jspa?threadID=676596&start=...) This problem seems underspecified, because it doesn't state whether you can iterate repeatedly over the sequence in question. If you can, it's easy. If you can't I'm not sure how to do it.
Re:Codility - Automated tests of programming skills
by samstokes on Friday January 08, 2010 @03:45PM
(#1040393)
I assumed that "sequence is very long" meant "the time (and space?) complexity of your solution matters". Iterating over the sequence x times doesn't change the time complexity, so long as x << n (i.e. it's O(n) whether x is 1, 2 or 3).
Unfortunately, revealing whether my assumption proved correct would give away how they assess the solution to this demo problem, which some might view as a spoiler.
EDIT: since people are posting actual solutions, I suppose mere analysis of the testing methodology is okay. My assumption was correct.
Re:Codility - Automated tests of programming skills
by bsaunder on Friday January 08, 2010 @10:04AM
(#1039644)
iterate repeatedly over the sequence in question
Does that ever seem like a good idea? The description suggests it may be a long list. That should give you a hint that you don't need to do this to solve the problem.
Re:Codility - Automated tests of programming skills
by anc2020 on Friday January 08, 2010 @03:13PM
(#1040309)
The best case possible is to iterate the list once - you sum the list from left to right holding cumulative answers and if the final sum is zero, that index is a valid answer.
The worst case of the best algorithm is to iterate twice - firstly as above, and then again in the opposite direction, comparing values from the first iteration.
Re:Codility - Automated tests of programming skills
by psranga on Friday January 08, 2010 @03:47PM
(#1040396)
Really nice site. Well done. The example problem that showed up to me (equilibrium point detection) was not trivial, but didn't require advanced stuff which a halfway decent programmer wouldn't use day in and day out.
I also like the way you tell me what you scored on and what my result was.
I wonder if you should relax the condition that you can submit only once.
I gather TopCoder offers something similar. What's your strategy to compete with them?
PS: got 94 using C++ in 16 min b/c I didn't check overflow of large numbers. Good thing you check for that.
Re:Codility - Automated tests of programming skills
by ThinkWriteMute on Friday January 08, 2010 @06:53AM
(#1039322)
You know, now that I think about it...The description of the test was pretty incoherent. If I don't know what my test data is, and if the problem description is incoherent, I can't really make it work.
It's also taking a metric fuckton of time to evaluate my solution. Which apparently fails. For reasons I can't debug because I can't see the test data.
Any company that relies on code snippets, incomplete problem statements and undefined test data to judge coders will get what they deserve.
Re:Codility - Automated tests of programming skills
by andrewcooke on Friday January 08, 2010 @07:34AM
(#1039362)
maybe there's more than one task, but the task i took appeared to be completely specified (the one where you find a balance point - i may have remembered the name wrong - for an array, so that the sum of elements to either side is equal).
it wasn't obvious to me what to do on first read, but when i paid a little more attention, everything was there (including edge cases, as far as i could see). in particular, they gave a formal numerical expression that was unambiguous (and which didn't seem to contradict the text).
Re:Codility - Automated tests of programming skills
by loup-vaillant on Friday January 08, 2010 @09:20AM
(#1039540)
The spec was clear, but the fact that the test dataset used for the test was the very one given as an example wasn't. plus, it was a bit frustrating not being able to fiddle with the test dataset. I would have done it locally with my compiler, but the necessary boilerplate would have lost me time. So, I didn't test enough, and made a silly error which went undetected because of the trailing zero. I still got 75%, but that looks quite miraculous.
EDIT: It appears I can customize my test dataset, and I didn't see it. Silly me.
Re:Codility - Automated tests of programming skills
by jakacki on Friday January 08, 2010 @07:25PM
(#1040786)
Yes, it is even possible to add your own data sets, Codility will tell you what you code returns for them. I guess we should make it more apparent in the interface.
Re:Codility - Automated tests of programming skills
by tsiki on Friday January 08, 2010 @09:47AM
(#1039598)
As a student this is very interesting since it could also be used to measure self progress. There's a myriad of ways to test your abilities in most subjects, but if I want to find out if I'm any good at programming or to measure my progress, there are very few easy ways to do that. Are you going to sell this to just business's?
Re:Codility - Automated tests of programming skills
by jakacki on Friday January 08, 2010 @07:08PM
(#1040761)
Codility tech team is from Warsaw University, we are more than familiar with TopCoder, ACM CPC, IOI etc. We deliberately keep Codility tasks at fundamental level. Codility is not a competition for 100 world's best programmers, it is a tool to tell decent programmers from poor ones. We can raise the bar arbitrarily high, if there is a customer who needs it.
Re:Codility - Automated tests of programming skills
by stonemetal on Friday January 08, 2010 @09:15AM
(#1039531)
For C++ it would be nice to know what compiler is being checked against. At least in the demo question I would use a 64 bit int to hold the sum if the arrays are really "large" but MSVC names the type __int64 vs G++ using long long and other such minor details. Same for the other languages aka python 2.x or 3.x etc.
Re:Codility - Automated tests of programming skills
by jakacki on Friday January 08, 2010 @07:21PM
(#1040779)
We want to avoid sticking to particular compilers/interpreters. An expert in MSVC should not be penalized only because we use GCC. Also we design the tasks so that they are compiler-agnostic (actually they are even language-agnostic). It is sometimes a bit tricky due to non-standardized type sizes (e.g. C, C++, PHP), but generally the tasks can be solved using very limited and portable subset of a language.
Edit: And won't this be a problem if interviewees can access the internet? The interviewers will have to make sure they can check the browsing history of the computer being used!
Re:Codility - Automated tests of programming skills
by anc2020 on Friday January 08, 2010 @10:03AM
(#1039641)
I don't mean the interviewers should shut off internet access entirely, but what I mean is that if they do allow access to the internet, they'll need to check the interviewee hasn't just googled for the solution to their codibility test (and they'll also have to make sure the interviewee can't delete their browsing history too).
Re:Codility - Automated tests of programming skills
by kbob on Friday January 08, 2010 @10:56AM
(#1039752)
Codility people, if you're reading, here's a bug report.
The test window's layout is fubar if you use a small window
size. I was using 800x600 and the language selection buttons were hidden behind the source code text box. Something (couldn't tell what) overlapped the first two lines of the instructions.
Re:Codility - Automated tests of programming skills
by alextgordon on Friday January 08, 2010 @06:47AM
(#1039314)
I'm not sure the timer is a good idea. For one, I found the seconds counting down very distracting. It also encourages quick and inefficient solutions that are easy to hack out, over more efficient algorithms that may take longer to code.
Maybe you should time the programs, instead of the programmers.
Re:Codility - Automated tests of programming skills
by marcin on Friday January 08, 2010 @06:50AM
(#1039318)
We are timing the programs as well - with the scalability assessment. Recruiter can change the timing anyway, but we feel that a bit of pressure is good. You don't always have comfy conditions on the job after all.
Re:Codility - Automated tests of programming skills
by dustin on Friday January 08, 2010 @10:03AM
(#1039642)
I also found the clock annoying. If someone does this for an interview they're likely to feel enough pressure anyways. I'd let the test taker hide it and maybe redisplay it automatically with 5 minutes to go.
As for comfy conditions on the job - I would hope that default conditions are comfy. When interviewing I'm more interested in how people will perform day-to-day rather than in rare situations. Most programming jobs don't really require employees to be "cool under pressure."
Other than that, good job on the execution so far. Reminds me a lot of TopCoder, just applied to a specific niche in a good way. I would even consider using this or recommending it to colleagues for hiring.
Re:Codility - Automated tests of programming skills
by loup-vaillant on Friday January 08, 2010 @09:27AM
(#1039553)
Right, but maybe not down to the second. I personally didn't mind, but counting seconds causes a distracting flickering. Maybe you should consider activating it on the last minute only ?
Re:Codility - Automated tests of programming skills
by anc2020 on Friday January 08, 2010 @10:07AM
(#1039653)
Might need some way of making explicit which libraries are available?
eg. if the programmer goes off and writes a multithreaded solution, they might only realise too late that pthread isn't included. Admittedly they should have checked first using verify, but you never know with interview pressure.
Re:Codility - Automated tests of programming skills
by jakacki on Friday January 08, 2010 @06:56PM
(#1040748)
Unless it is stated explicitly, we do not provide anything extra but the vanilla programming language and standard libraries. It is more clear for some languages (e.g. Java, C++) than other (e.g. PHP, JavaScript), but we try to stick to the reasonable canonical set of libraries.
Re:Codility - Automated tests of programming skills
by theycallmemorty on Friday January 08, 2010 @10:17AM
(#1039673)
Seems like a cool idea. If they were smart they'd not only target companies looking for people but people looking for jobs. e.g. Adding a "Allow the codility community to take this test" check box. Then you have the a whole world full of people taking your test.
Re:Codility - Automated tests of programming skills
by kkowalcz on Friday January 08, 2010 @01:03PM
(#1040036)
Actually it's there - if you click on "Take a free demo test" (http://codility.com/demo/take-sample-test/) on the Codility page there is a free test and you will be given the option to tweet your result at the end.
Re:Codility - Automated tests of programming skills
by olliesaunders on Friday January 08, 2010 @07:55AM
(#1039392)
Yeah, that demo problem would probably take me at least several hours to solve. I'm bad with sequences and I have a volatile short term memory which means I make mistakes and am slow to process numbers and logic. Interestingly these flaws cause me abhor any over complex logic or code where you have to remember more than 4 or 5 things at once to use properly, which makes my code clean. So maybe I'm not a good programmer (I can't write the complex stuff very easily) but the code I write is usually good. Try testing for that Codility!
Re:Codility - Automated tests of programming skills
by Kliment on Friday January 08, 2010 @05:45AM
(#1039252)
Very nice product. Much-needed quantification in tech recruiting. Maybe consider marketing it to the stackoverflow people. They have been running a job board with much fanfare since recently. (94 points for me in 7 mins, yay)
Re:Codility - Automated tests of programming skills
by andrewcooke on Friday January 08, 2010 @06:01AM
(#1039267)
also thought this was nice after trying it out.
one thing i missed, though, was the ability to write and run my own tests. perhaps a "scratchpad" that evaluates code and returns a result would be a nice addition?
Re:Codility - Automated tests of programming skills
by marcin on Friday January 08, 2010 @06:46AM
(#1039313)
Sorry, some bug on our end - it should be there. We are fixing it right now. In the meantim - if you sign up for a trial account [http://codility.com/accounts/codility-register/] it works fine there (a blue "add a test case" button in the bottom left).
Re:Codility - Automated tests of programming skills
by andrewcooke on Friday January 08, 2010 @07:36AM
(#1039369)
thanks. another smaller point i just remembered - i was a bit surprised to see that the evaluator could see my submissions to the "validate" function. given that you do say they won't influence the score i assumed that they would be discarded.
i can see there are several reasons why this is useful to the evaluator, and i can also see that you're getting close to saturating the user with instructions, but in a perfect world it would be good to have "full disclosure" (i guess you do, in that anyone can run the example....)
I have no idea where you saw this being implied. The software highlights syntax and lets you check your code as many times as desired, as well as giving hints if your result isn't correct and providing compiler output. Nothing stops you from googling something. It's not like the software will destroy your file and egg your house because your algorithm is o(n2) instead of o(n); that's up to whoever is evaluating the results.
The fact that it has a polish and chinese version should also give you a little hint about the intended audience.
That said, as soon as the test popped up, I opened my eclipse, wrote the code, cut & pasted .. viola! (in python btw)
It would be nice if the boundary conditions were better specified (people seem to have been bitten by overflowing integers). This could be inferred if you told people that you're using a 32-bit machine (the sum of elements is at most SIZE_MAX * INT_MAX; if int and size_t are 32-bit, SIZE_MAX * INT_MAX < INT64_MAX.)
I was also somewhat surprised to see that my code compiles without #include <stdint.h>. I'm not sure this is correct (it could be solved by putting the code written by the candidate before the test code).
Finally, I assume you have proper sandboxing set up?
#include --- good catch! We are not very strict about missing includes, but this is definitely not something we want to penalize.
Sandboxing -- absolutely! :-)
I got 94 using C in 12min (didn't check for overflows of large numbers, silly haskell making me used to Integers).
I'd give people access to their language's output facility for playing around to the solution, but that is a minor nitpick. (And I suppose I could open an irb instance on my machine for exploration and then give you code when I'm good and ready.)
My Ruby solution, in case we've got anybody who is struggling: http://pastie.org/771438 This would be my first cut if you asked me to whiteboard this.
The site's very slick. My only feedback is the histogram on the results page could be more prominent - it's cool, useful information but I ended up inspecting the html elements to work out what it was.
Also the results page seemed to keep refreshing for me while waiting for results (Safari 3). Surely that should've been AJAX'd?
Original: http://pastebin.com/m6e742f56 Parallelised: http://pastebin.com/m32a60b1
In face-to-face interviews you'd probably follow up by adding constraints: "your algorithm uses O(n) space in addition to the input, can you do it in constant space?". It would be cool if Codility supported that kind of workflow - following up the submitted solution with refinements on the original question. (Relatedly - do you support the workflow where you get a second try if you didn't get 100, maybe because the evalution revealed a constraint that wasn't clearly stated in the problem outline?)
http://www.cdolivet.com/index.php?page=editArea
very nice project on LPGL license.
http://twitpic.com/x6wkq and even the code matches line by line I guess - http://twitpic.com/x6wnp
> Hmm so Codility's editor is copy of the one SPOJ uses
Please don't flame. Codility has not copied it from anybody, its EditArea-based interface is in place since 2008.
Hmm so Codility' and SPOJ use a similar editor, so which one do they both use? Can I too get it :)
Thanks for pointing out. /me looks into EditArea
https://bespin.mozilla.com/
Edit: just noticed that you'll have to wait a few days to get an embedded release that has syntax highlighting ( http://groups.google.com/group/bespin/browse_thread/thread/0... )
best Marcin Codility
You might want to lock down functions like glob() and readfile() in PHP -- they effectively give the coder read access to your file system.
Unfortunately, revealing whether my assumption proved correct would give away how they assess the solution to this demo problem, which some might view as a spoiler.
EDIT: since people are posting actual solutions, I suppose mere analysis of the testing methodology is okay. My assumption was correct.
Does that ever seem like a good idea? The description suggests it may be a long list. That should give you a hint that you don't need to do this to solve the problem.
The worst case of the best algorithm is to iterate twice - firstly as above, and then again in the opposite direction, comparing values from the first iteration.
It isn't that difficult to solve, but I like the idea.
I also like the way you tell me what you scored on and what my result was.
I wonder if you should relax the condition that you can submit only once.
I gather TopCoder offers something similar. What's your strategy to compete with them?
PS: got 94 using C++ in 16 min b/c I didn't check overflow of large numbers. Good thing you check for that.
It's also taking a metric fuckton of time to evaluate my solution. Which apparently fails. For reasons I can't debug because I can't see the test data.
Any company that relies on code snippets, incomplete problem statements and undefined test data to judge coders will get what they deserve.
it wasn't obvious to me what to do on first read, but when i paid a little more attention, everything was there (including edge cases, as far as i could see). in particular, they gave a formal numerical expression that was unambiguous (and which didn't seem to contradict the text).
EDIT: It appears I can customize my test dataset, and I didn't see it. Silly me.
http://codility.com/demo/results/?id=demoMPC2W9-JDG
http://pastebin.com/m6e742f56
Edit: And won't this be a problem if interviewees can access the internet? The interviewers will have to make sure they can check the browsing history of the computer being used!
The test window's layout is fubar if you use a small window size. I was using 800x600 and the language selection buttons were hidden behind the source code text box. Something (couldn't tell what) overlapped the first two lines of the instructions.
Firefox 3.5.7/Ubuntu Linux 4.10, FWIW.
Maybe you should time the programs, instead of the programmers.
As for comfy conditions on the job - I would hope that default conditions are comfy. When interviewing I'm more interested in how people will perform day-to-day rather than in rare situations. Most programming jobs don't really require employees to be "cool under pressure."
Other than that, good job on the execution so far. Reminds me a lot of TopCoder, just applied to a specific niche in a good way. I would even consider using this or recommending it to colleagues for hiring.
Consider making the clock like on http://www.sudokuslam.com/ - with an icon right next to it to show or hide it.
eg. if the programmer goes off and writes a multithreaded solution, they might only realise too late that pthread isn't included. Admittedly they should have checked first using verify, but you never know with interview pressure.
That said: Jesus I must still suck, or maybe Math simply isn't my strong suite.
BTW 10-20% of the population is like me.
I readily concede that coding ability isn't the only thing a dev needs, but it is absolutely required for most jobs.
I took a demo quiz and it worked fine. Great idea for a product.
However I think I'm too competent for the test because I was able to run something I probably shouldn't have been able too.
I've already sent it around at the company I work for. I'm hoping we can put it to use.
one thing i missed, though, was the ability to write and run my own tests. perhaps a "scratchpad" that evaluates code and returns a result would be a nice addition?
My bottom left corner is empty. Except when I click "verify" it shows message whether the tests passed or failed.
But I don't see the tests themselves, nor did I notice a way of writing my own. I've tried it in Firefox, IE 8 and latest dev Chrome.
So, is it possible to write my own tests when I'm taking the test, or only when I'm creating a new one?
i can see there are several reasons why this is useful to the evaluator, and i can also see that you're getting close to saturating the user with instructions, but in a perfect world it would be good to have "full disclosure" (i guess you do, in that anyone can run the example....)
Anyway, I like it a lot. It sure beats writing tests in front of a blackboard.