And how about this lovely question:
> A personal identification number that opens a certain lock consists of a sequence of 3 DIFFERENT digits from 0 though 9, inclusive. How many possible PINs are there?
Putting aside the value of this question in the first place, the correct answer is 720 but it also includes the answer 1000 in case you didn't notice the emphasis on “different”.
Doesn't seem like a great test to me unless it's about your reading comprehension and ability to decipher code with single-letter variable names.
Really? I mean, to first approximation, all software engineering work is debugging. And to first approximation, all debugging is about carefully reading results and the code that generated them. That sounds pretty spot-on to my eyes.
And the variable naming quip... are you serious? One of the worst mistakes you can make when trying to read code for understanding is to rely on documentation layers like comments and variable names. Those things lie. Read the code.
I mean, obviously people do rely on those things for understanding, so when writing code it's important to have good semantically and contextually clear names for stuff (which certainly can include single letter names for local symbols repeated often!). But for debugging, you really have to train yourself to read the code, not the names.
I don't really agree that these questions test debugging skill. In real life if I read a bit of confusing code and get tricked, I will usually know immediately thanks to my IDE, compiler, automated tests, debugger, or behavior of the program. On this test if I get tricked, I just lose points.
In my experience most debugging presents as trial-and-error (a terrible test-taking strategy) or seeing a bug and reasoning at a high level which component of the software is responsible and gradually narrowing it down. Carefully reading through lines of code is usually the last step, unless I've got an especially elusive bug.
> And the variable naming quip... are you serious? One of the worst mistakes you can make when trying to read code for understanding is to rely on documentation layers like comments and variable names. Those things lie. Read the code.
I'm not sure what sorts of code you're reading but this opinion seems extreme. Do you really just wholly ignore comments and variable names? I have encountered inaccurate examples of both but I have never thought to distrust them by default. As far as I recall this has not caused me serious issues while debugging.
I don't think I agree. You're talking about heuristics for debugging code and I'd argue that many of us apply similar heuristics to reading comprehension. Computer Science is not the same thing as reading comprehension, but I have to argue that if you're really good at one of these; you're probably potentially good at the other.
Being able to go from some constructed story to a math problem is a skill that can be tested elsewhere (and really, should be assumed if someone clearly has proficient enough numeracy to do combinatorics) Questions should be testing one thing as isolated as possible, not the same things repeatedly.
I can see this in my students in my introductory class. You can ask them to calculate a concentration of a solution and they will reply just fine. But when you ask them to go into the lab and actually prepare a solution of a certain concentration you draw blanks in 9 cases out of 10. Students are trained in AP Chemistry to spot problems according to keywords and apply the appropiate formula, but they have never thought about the underlying meaning of the things they do, and they cannot connect different subjects to save their lives.
The mechanical kinesthetics of combining things is not the same skillset as the mathematical process of computing it. Lab work is its own craft.
Based on what you're saying, I'm assuming you already put your questions in the "Story time" format and are getting that 1/10 success rate because the story problems are yet a Third skill - this time one about reading comprehension and language mastery.
Reading comprehension isn't kinesthetics and that's not computation. One may partially inform the other, but each skill is fairly independent.
The sample questions cover amortization, memory allocation, finite state machines, formal logic... If you just studied enough to pass a whiteboard interview, I'd hazard that you probably can't answer half of these questions without looking up what the symbols mean.
Sure, most people here may be able to figure out the PIN question, but it's not about whether any single question is easy or hard, but whether the test can sort the students according to ability. A good student may misread one or two, but a bad student is going to miss a lot more.
What this really measures is corricilum. And pressumbly it's more stuff like this in the US then Russia?
If a CS student taking this test does not get “tricked” by this question, they should definitely be able to answer it.
Does anyone have an assessment of reliability/validity/fairness of multiple choice comparing to other test types? (afair objectivity is very good in multiple choice tests)
For a fair comparison, the translator would have to fully appreciate the relevant connotation of every significant word like "different" in every question, and successfully convey the same in Russian. You couldn't just go to a regular translation service and get a good result.
Although I have no faith in a multiple choice test as a predictor of CS knowledge anyway. That is just a horrible test format anyway.
I did not get 100%. I actually thought the exam was pretty good, but it tested something like 1% of the skills I use as a working programmer. Skills like API design, software architecture, documenting code, and yes even naming variables, which I think are important to writing quality code.
It can be easy for people to forget that many people (even coders) have never done this type of math before.
The first time you pick, you have 10 different numbers to choose from, the second time, 9, the third time 8.
1000 = all possible combinations from 000 to 999.
less 10 combinations with all numbers identical is 990.
now there are three patterns for numbers to be identical left that need to be taken out: 00x 0x0 x00. The x can be replaced with all digits != 0 in this example, so thats 3 (patterns) x 9 ('x' digits) = 27. That times 10 digits that can have double patterns is 270.
1000 - 10 - 270 = 720.
import itertools
print(len([x for x in itertools.product(range(10), repeat=3)])) #1000
print(len([x for x in itertools.permutations(range(10), 3)])) #720
The reason it's less is because each number has to be different, so you can't have 000 or 111, etc.
The formula is:
n! / (n − r)!
Since there is 10 (n=10) total numbers, 0-9. And it can only be 3 (r=3) digits long. Then it would be,
10! / (10-3)!* which would result in,
10 * 9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 / (7 * 6 * 5 * 4 * 3 * 2 * 1) = 10 * 9 * 8 = 720.
for j = 1-10
if i == j break
for k = 1-10
if i == j or i == k break
log(i,j,k)
(I don't understand HN formatting still)
#!/usr/bin/python
r = [(i,j,k) for i in range(10) for j in range(10) for k in range(10) if i!=j and i!=k and j!=k]
print(len(r))Do people really struggle with problems like these?
Don't use that same password for anything else!
Different can be interpreted in multiple ways.
Reminds me of a physics test where the question is something like:
Susie is almost at her house and comes to a stop in 4 seconds. What was her acceleration.
Anyway - IMO this is a terrible question.
>exam that was developed by the U.S. Educational Testing Service
Who could have guessed that?
I switched into programming and am low paid, and am doing this right now for interviews. Practicing for whiteboard interviews, or coding tests (real time online and in person).
I re-read the language docs and then did (so far) 20 or so self-timed programming tests. Most of them I did in 5-30 minutes, 1 or 2 I got stuck on and subsequently reviewed. After finishing reading the language docs I will read though the platform docs, and may do some quizzes on that as well. As well as whiteboard high level class creation tests (HAS-A, IS-A).
The tests have a lot of array stuff, String manipulation, using long instead of int, casting from one type to another, doing a lot of stuff with maps and sorting maps by value. It also uses things then can be done quickly and compactly in streams. I do these things once in a while, but now have it memorized and down cold.
Hopefully this all leads to the latter result you mentioned.
Also, what do people think about triplebyte for this?
This annoys me to no end, since ABET is used to gate certain federal jobs and the Patent Bar, even in Computer Science.
However, every school you listed is ABET accredited in other engineering disciplines (mechanical, chemical, civil, electrical). So ABET-accreditation is important to some fields.
I don't want to be too elitist, but where would you like to recruit reviewers for your computer-related patents from: DeVry Tech or Stanford? Do you really want to actively exclude Princeton and Yale grads from working on software that runs the government?
I get the appeal of having a standardized 'seal of approval', but for Computer Science, ABET is clearly not that. It looks like several other disciplines are also moving away from ABET (e.g., https://cen.acs.org/articles/95/i48/time-leave-behind-chemic...)
Acceptable Computer Science degrees must be accredited by the Computer Science Accreditation Commission (CSAC) of the Computing Sciences Accreditation Board (CSAB), or by the Computing Accreditation Commission (CAC) of the Accreditation Board for Engineering and Technology (ABET), on or before the date the degree was awarded. Computer science degrees that are accredited may be found on the Internet (http://www.abet.org).
[page 6 of this pdf: https://www.uspto.gov/sites/default/files/documents/OED_GRB....]
A lot of fed jobs have engineering-centric requirements. This one, for example, wants an accredited degree or experience under a PE....to do network management. https://www.usajobs.gov/GetJob/ViewDetails/525193800
Can you clarify what this means?
There is a huge variation among countries in how subjects are taught and tested. I would like to add that many of my peers have resorted to streamlining for whiteboard interviews, and unsurprisingly, they end up with well-paying jobs.