Re:U-M finds students with alphabetically lower-ranked names...
by barfbagginus on Tuesday April 23, 2024 @12:31AM
(#40128536)
I don't think they're misunderstanding randomness.
They are claiming that randomness is not sufficient for fairness.
And I agree. Adding some determinacy to the shuffling design would reduce the variance of unfair scores that students receive. Ie, it could reduce the likelihood that a student's scores will be biased by a run of advantaged or disadvantaged scoring orders.
It wouldn't be very hard to design such a scheme. It would produce fairer results in the short run. And in the long run, it would converge to the same fair averages as the fully random process.
In fact it would converge faster. This short run statistical efficiency matters. Students don't have access to the long run results - since they're evaluated in the short run.
It's also worth noting that randomization in a context like this is inherently an imperfect solution to a problem that generally can't be solved perfectly. If we find out that weird ordering biases exist, I think randomization is done on the assumption that many we don't know about could also exist, that there's no clear way to mitigate them completely, and then randomizing the order per-instance is just the best we can do to ensure it's fair (Which, again, won't be perfect. Perfect isn't available)