News for hackers. Stuff that matters.
Re:U-M finds students with alphabetically lower-ranked names... by madeofpalk on Saturday April 20, 2024 @12:21PM (#40098470)
> you have to shuffle the order in which you grade these submissions each week, for fairness

I don't think this is fair. It's just a more randomly distributed unfairness, rather than by a deterministic factor (like the student's name)

'Fair' would be each student is assessed independently for the work they did, rather than their mark being impacted by how early or late they were marked.

Re:U-M finds students with alphabetically lower-ranked names... by shepherdjerred on Saturday April 20, 2024 @12:35PM (#40098554)
It would be essentially impossible to have something "truly" fair for open-ended questions since humans are stateful.

Maybe this is a case that AI could actually do quite well.

Manually grade the answers and identify the classes of mistakes. Then hand the classes of mistakes to the AI and ask for it to determine which answers have which types of mistakes.

Once you've done that, you just need to associate a deduction for each type of mistake and do some simple math.

Re:U-M finds students with alphabetically lower-ranked names... by vagrantJin on Saturday April 20, 2024 @05:57PM (#40101369)
what do you mean AI? you must be joking.
Re:U-M finds students with alphabetically lower-ranked names... by shepherdjerred on Saturday April 20, 2024 @06:26PM (#40101574)
Imagine a question: compare bubble sort and quick sort algorithm.

Some students might mix up the algorithms, some might give the incorrect computation complexity, some might describe them incorrectly in some way.

Manually grade some (or all of) the answers by noting the kinds of things students got wrong (e.g. the above criteria). Then, feed in to ChatGPT (or your favorite alternative) the answer + the categories of mistakes to expect.

Here's a simplified example: https://chat.openai.com/share/bf801e12-51d5-4255-9968-bbf91b...

Re:U-M finds students with alphabetically lower-ranked names... by luplex on Saturday April 20, 2024 @12:25PM (#40098498)
There are many notions of "fairness", many of which are logically incompatible with each other.

In this example, I think it's kind of fair to give everyone an equal chance of being advantaged. You're not hurting anyone specifically.

Re:U-M finds students with alphabetically lower-ranked names... by gqcwwjtg on Saturday April 20, 2024 @12:42PM (#40098610)
Is that distinction worth making here? There’s no way to “assess independently” the work of each student without some amount of randomness. But I think that’s okay, because isn’t randomly distributed unfairness just… fairness?
Re:U-M finds students with alphabetically lower-ranked names... by jcparkyn on Saturday April 20, 2024 @08:37PM (#40102286)
I think an important difference is that when you shuffle them, the unfairness stops being correlated across multiple assignments, so the "aggregate" unfairness over the course of the semester is much lower.