News for hackers. Stuff that matters.
Re:CheatGPT by Al-Khwarizmi on Monday February 20, 2023 @07:06PM (#34874658)
As a CS professor myself, I don't think ChatGPT really creates entirely new problems for assessment, but rather makes problems that we had been sweeping under the carpet more obvious and harder to ignore.

Any assignment that can be cheated by using ChatGPT could already be cheated before by asking a friend, an expert or paying someone else to do it.

But most teachers assumed this doesn't happen often, and thus acted as if it wasn't a thing. Making life much simpler and easier.

Facing ChatGPT will probably help us make assessment fairer in the long run (and the path is probably going to be a lot of traditional on-site assessment, abandoning the continuous assessment fad that has always been rather disastrous for equity, even if again, life is easier ignoring this).

Re:CheatGPT by kurthr on Monday February 20, 2023 @07:22PM (#34874785)
As a first year grad student, I had a professor in material science who gave the same assignments and mostly same exams every year. Of course a) I didn't know this b) using previous works was prohibited c) cheating was rampant. I only found this out after getting heavily marked down on the first HW and joining a study group which was moderately chaste (eg only used the answers to check our work before turning it in). Missing a single question on HW or the exam would drop your grade a letter, and three or more was likely D/F range. When I brought this up with the professor before the final he at least acted completely shocked as I asked how likely he thought it was that half the class never missed a question (yes half the class got As). Really weird conversation, but most of the class was undergrads, and I think they liked the easy grade with minimal work and so he got 'good ratings'.

Undergrad college had 'infinite time' and 'open everything but people' exams, but graded on a curve so that sometimes getting half the questions right was an A. Much more stressful, learned more, and basically no cheating.

Re:CheatGPT by Aperocky on Monday February 20, 2023 @08:49PM (#34875474)
Learning over the centuries has always taken a character of reciting information, since the ability to do so was valued at the time.

With the technologies of today, the ability to recite knowledge (versus using said knowledge when provided, and the ability to look for said knowledge) is rapidly losing value. But our education system has not moved on and are mostly stuck at using recital to judge student capability.

A complete rethinking of how to assess student performance is long overdue. For subjects that cannot be assessed without recital, maybe rethink the necessity of existence of those subject for teaching.

Re:CheatGPT by andsoitis on Monday February 20, 2023 @09:32PM (#34875757)
> With the technologies of today, the ability to recite knowledge (versus using said knowledge when provided, and the ability to look for said knowledge) is rapidly losing value.

I didn’t think you are implying that having knowledge (ready to go) is of no value.

Having more knowledge gives a huge advantage in time and space. Not only can you solve problems much faster (you don’t need to find, read, understand, and internalize the “knowledge”) you can also make connections that others would miss because to gain knowledge is deeper than mere recall.

Re:CheatGPT by __MatrixMan__ on Monday February 20, 2023 @09:53PM (#34875872)
I got my first degree in 2009. Recently started again. Took a biology test today and was pleasantly surprised how non-recital it was.

It focused on a planned development (hotels, grocery stores, etc) in a part of town that everybody in the class is quite familiar with and asked us to apply concepts from lecture to predict the impact to a given set of species.

It was masterful. Hit all the points, anchored in our everyday lives, applied in a way that rote memorization just wouldn't help with. I had a moment like: WTF I've lived here my whole life and never seen a wild ferret. Searched it after the tests and turns out they're native and critically endangered.

Just saying, the good ones are out there.

Re:CheatGPT by jonnycomputer on Monday February 20, 2023 @11:30PM (#34876604)
When I realized that there were numerous copies of past exams floating around, I just made them available to everyone equally. Of course, I always wrote new exams, but it is helpful to see what kinds of questions you may be asked on an exam.
Re:CheatGPT by sq_ on Monday February 20, 2023 @11:35PM (#34876648)
> Undergrad college had 'infinite time' and 'open everything but people' exams, but graded on a curve so that sometimes getting half the questions right was an A. Much more stressful, learned more, and basically no cheating.

I tend to really appreciate 'open everything but people' exams, since they're a good way to actually check that people know how to complete a task. I think that a good one is probably harder for a professor to write, but the ones that I've taken have been a much better test of my understanding of the topic than ones where I was concerned with memorizing what the API for POSIX threads was.

Not so sure about infinite time, though. Seems like it makes it really easy for someone who has the time to dedicate a whole day (or the obsessiveness to do it even if they don't have the time) to do way better than someone who, for whatever reason, can't.

To me, the gold standard for computer science exams would be ~90m-3h with an open everything policy, including both a "do you know/can you apply the theory?" and a "can you do something practical with that?" section.

Re:CheatGPT by lobocinza on Wednesday February 22, 2023 @06:41AM (#34894289)
Reminds me of a girl from my chemistry lab group on college that replaced our experimental findings at the last minute without telling us because ours was drastically different than the "answer" from past semester works. The experiment was to identify a substance using it's density or something like that. She didn't stop to think that maybe, just maybe, the professor used a different substance on that semester. That bothered because while I didn't take my graduation seriously I really cared for rigor and measured things twice.

Cheating was prevalent in certain situations. I had a teacher that would leave the room during exams leaving the students to do whatever they wanted, which included the whole class hot debating the answer of each question. An engagement level that would never occur in a normal class.

In the CS lab (C++ for engineers) it was common for the student that finished the assignment first to mail the code to the whole class. Most would simply copy it, leave early and call it a day. Many also struggled with programming and put it in the category of "things that are hard and will not need to know".

Re:CheatGPT by jraph on Monday February 20, 2023 @07:12PM (#34874713)
> the continuous assessment fad that has always been rather disastrous for equity

Can you expand on this?

I lived the two models as a student and still can't make up my mind on which is best. I feel like it just takes catching some bad cold the week of the exams or a bad exam subject / topic to fail them which seems pretty unfair, but I also know my shit better and have more perspective the week of the exam and prefer not having to stress every week because of the continuous assessment.

Re:CheatGPT by arvidkahl on Monday February 20, 2023 @07:16PM (#34874748)
There is also an option outside "one exam" and "many exams" — and that's "no exams." Obviously, this is spectacularly hard to fit into the current system, but you see this a lot in vocational schools where project-based work makes up the majority of the grade and exams are held to comply with regulatory requirements.

ChatGPT might just fit into an educational world that values outcomes over fact retention. One may dream.

Re:CheatGPT by MengerSponge on Tuesday February 21, 2023 @02:21AM (#34877843)
Well-structured exams are the single most effective teaching tool available. Assessment is a fringe benefit. I'm a strong advocate for "even more exams", where you take the highest score on particular topics.

There are variations, but if you're curious check out "Standards-based grading" or Mastery-based grading. It's in a weird and shitty limbo where a lot of primary and secondary schools have adopted it partially with inadequate training/support, and it's also getting fucked by our shitty publishing industry, same as the botched common-core rollout.

Re:CheatGPT by nemo44x on Monday February 20, 2023 @07:36PM (#34874900)
They’re saying that people that were allowed into their programs/universities because of quotas and not merit struggle to prove competency through assessment.
Re:CheatGPT by fidgewidge on Tuesday February 21, 2023 @01:25AM (#34877442)
Wouldn't that be an issue with point in time assessment too?
Re:CheatGPT by nemo44x on Tuesday February 21, 2023 @07:36AM (#34880396)
How so? When you’re admitting under powered and under prepared people into a program then they will fail. Unless you dumb down the assessment and curriculum. Which universities have done to make their numbers achieve equity.
Re:CheatGPT by fidgewidge on Wednesday February 22, 2023 @04:20AM (#34893419)
Yes but I don't quite get the difference between exams and coursework here.
Re:CheatGPT by Al-Khwarizmi on Tuesday February 21, 2023 @05:12AM (#34879150)
This is probably an unpopular opinion, but in my view, traditional final exams tend to be a great equalizer. Continuous assessment is affected much more by personal circumstances.

For example, continuous assessment tends to force students to devote a constant amount of time throughout the course, which is fine for those who can afford to be full-time students, but disastrous for students who need to have a job to pay the bills. I have seen plenty of students like that getting frustrated because they lose points to this.

On the other hand, continuous assessment typically relies on assignments at home and these are deeply unequal and classist as well. The difficulty of these assignments depends on whether the student has someone more experienced to ask, whether they are paying for a private tutor, or whether they can plainly buy the assignment. I have known cases of students that basically bought every project and assignment that could be done at home, you know what made them sweat because it couldn't be bought? The final exam.

But almost no one cares about these things, continuous assessment is seen as the "progressive" thing to do because it's more modern and different from tradition, even if it's extremely classist and disproportionately harms working-class students.

On the other hand, it is true, as you point out, that the final exam model disadvantages students who have the bad luck of getting sick in that week. It's a real drawback, sadly. I still think it's better all things considered, for the reasons above.

Re:CheatGPT by jraph on Tuesday February 21, 2023 @08:53AM (#34881131)
Okay, totally makes sense. Thanks!

I wish people didn't have to work during studies. That's one of the biggest reasons of failing.

Re:CheatGPT by makstaks on Monday February 20, 2023 @07:19PM (#34874767)
As a parent, I'm hoping this will reduce the at-home workloads and shift more work towards school. What's the real life value of having students go to school and then spend all-nighters trying to do homework and all the expected extra-curricular?
Re:CheatGPT by KSteffensen on Tuesday February 21, 2023 @04:54AM (#34879033)
As I remember my education time from high-school onwards, all-nighters were usually a consequence of slacking off during school time.

In college/university a fairly big part of the education was learning what was compulsory reading and what was optional. There was always more reading than hours in the day.

Re:CheatGPT by yodsanklai on Monday February 20, 2023 @09:20PM (#34875672)
> going to be a lot of traditional on-site assessment, abandoning the continuous assessment fad

Maybe because I'm from a latin country, but I was always under the impression that only on-site assessment mattered, as continuous assessment provides no signal considering many students cheat. Even what we call "continuous assessment" is done on site.

Re:CheatGPT by whazor on Tuesday February 21, 2023 @12:48AM (#34877201)
For my university, not that long ago, it was possible to finish computer science, without touching a computer.

Still in my time most exams were done on paper. Think of algorithms in pseudocode. But also a lot of logic like reasoning, with process automata, or relational algebra (databases).

Re:CheatGPT by belter on Tuesday February 21, 2023 @03:56AM (#34878559)
The graduates from your University must kill it on whiteboard interviews :-)
Re:CheatGPT by raydev on Tuesday February 21, 2023 @02:37AM (#34877972)
> asking a friend, an expert or paying someone else to do it

Surely you understand ChatGPT is infinitely more scalable. How many people can just "ask a friend" or have a spare money to pay someone else multiple times?

Re:CheatGPT by js8 on Tuesday February 21, 2023 @04:09AM (#34878662)
There is also a risk associated with asking another person to be an acomplice. Unlike ChatGPT, they might turn on you.
Re:CheatGPT by Al-Khwarizmi on Tuesday February 21, 2023 @05:15AM (#34879178)
That's exactly what I'm saying, right?

The problem has always been there but it was largely ignored because it wasn't that frequent (although I think it's definitely more frequent than most people make themselves believe).

Now that everyone has a "friend" to ask thanks to ChatGPT, people begin to care. How dare the plebs have the same options to cheat as privileged students!