News for hackers. Stuff that matters.
Re:Commercial jet collides with Black Hawk helicopter near R... by perihelions on Thursday January 30, 2025 @09:32AM (#42878037)
Isn't it a reasonable filter? I'd (naively) assume that all things being equal, a person who passes a test on the first attempt, is more likely to have a higher innate ability than one who doesn't.

Is this wrong?

Re:Commercial jet collides with Black Hawk helicopter near R... by michaelt on Thursday January 30, 2025 @10:40AM (#42878744)
That depends on the test, and the training surrounding it.

If I'm hiring for a job where there's a strict requirement to be 6 feet tall and there's nothing candidates can do to get taller - then once you've measured under 6 feet there's no point in re-testing you.

If I'm hiring for a job where there's a strict requirement that you be able to lift a 90 lbs weight, a lot of people will fail - but if they hit the gym, in a month or two they'll be able to pass easily.

If I'm hiring for a job where there's a strict requirement that you be able to lift a 90 lbs weight, but before the test every applicant goes through a 6 month strength training program, so we test them at absolute peak strength? Re-testing would be a waste of time, except in a few cases like if someone had a loved one die just before the final test.

If I'm hiring for a job with a written test, the test draws a random selection of questions from a question bank, and some questions are much easier than others? There might be a substantial random element to the results. If re-testing people produces substantially different scores, probably that indicates I need to improve the design of my test.

Re:Commercial jet collides with Black Hawk helicopter near R... by UltraSane on Thursday January 30, 2025 @09:46AM (#42878197)
Imagine if they only let people take the bar exam once.
Re:Commercial jet collides with Black Hawk helicopter near R... by cutemonster on Friday January 31, 2025 @01:58AM (#42885257)
Yes, wrong I think.

People's performance varies from day to day, and if doing the test just once, then some people will pass it because of good luck shape that day, although another day they would have failed.

When constantly predictable high performance is important, it makes sense to do the test on a bunch of different occasions, and look at the overall result (like, lowest, median and average scores). And it'd be ok to do a bit worse on one occasion, as long as you were always above the minimum.

Rather than in effect randomly picking just one test result.

Looking at the results from many tests, reduces false negatives, but also false positives.

(And of course if studying and practice matters, getting to re-take the test some years later: yes of course, but possibly the test should then be a little bit longer.)