https://www.insider.com/viral-tiktok-student-fails-exam-afte...
I don't much like such software, but I accept the current need for it. The problems people are describing seem to be faulty staff behaviour, not software.
"I was on the second question of an exam composed of 45 questions and I got a black screen. I'm still waiting to see what's going to happen as this is my final......... I tried to access the exam but was locked out and when i went to the exam the chat option wasn't available either. I emailed my professor and unfortunately, he wasn't watching the exam at the time and couldn't do much to help me out."
https://www.reddit.com/r/UBC/comments/g2ub05/god_kicked_out_...
The overall outcome (and the student's experience and stress) is ultimately a staff issue.
The professor should have been on call, and should have been able to get the student back into the questions, or, failing that, to offer another sitting. I have done both of these, among other remedies, and setting student expectations ahead of time is crucial for avoiding massively stressful problems (stuff like "If something goes wrong, stop and email me. I can add lost your time back and get you into the test again, no matter how long it takes to sort out. You won't lose marks from this." and actually follow through). Software inevitably fails sometimes, and it is just a tool. Staff need to make their own decisions.
If we agree that the current situation is so exceptional that we need to fully monitor all students taking exams (and that's still a big if, I also am a university teacher, we don't use any monitoring system. There are other solutions as well such as taking oral exams...), one could simply hire a bunch of proctors who watch the video feed. Considering the savings that universities have made they could send one to everyone's home even.
> I see it as a currently-necessary annoyance, as the least bad option. The alternatives have greater deficiencies: human invigilators using Zoom etc. don't scale (institution experience); oral exams don't scale (my experience moderating such assessments); no invigilation leads to cheating (by few students but enough to be a real problem, especially for professional qualifications - my direct experience and institution experience); shutting down education until the pandemic's over is unfeasible.
Last year, we tried not monitoring, we tried Zoom invigilation, we tried orals. Lockdown rules prevented us from sending in-person invigilators in one semester (and we couldn't have hired even 10% of what that would take) and the other we invigilated in person like normal. We are spending more than ever before, so there are no savings. All of these methods failed. This year, grudgingly, we have moved to proctoring software.
Regarding scaling of oral exams, there's actually some interesting research/calculations (I try to find the reference later) and the cross-over is somewhere around 150 students when oral exams become slower (I do think this is quite teacher and subject dependent though).
Also about budgets, I believe that your department is spending more than ever, the issue is property services departments of the universities should be saving large amounts. Considering lock-downs and staff and students working from home, maintenance cost should be way down. I suspect though that money lands in completely different buckets (don't get me started on the business of university property services, we pay rent in our overhead cost when we get grants that are higher than renting office space on the main shopping street in our city).
The oral exams I moderated didn't seem like they could scale past 20 students. The reason our online students are online is almost always because they are overseas, and the majority of them have limited English. In that context, orals are discriminatory, stressful, and very slow.
I'm in NZ. Our universities have mostly been operating in person as normal (i.e. costing just as much) with far fewer international students (who bring in money), except in addition we have comparatively small numbers of resource-intensive online students as well, and are attempting to bring parts of entire degree programs online for the first time.
Just because this solves your problem (it clearly doesn't) doesn't mean it's a good idea. Killing half the students also solves your problem.
You need to live in a world where this is not an option, period. What's the next solution then? People forget there are always more manual solutions even if they take more time...
Could you explain further why this did not scale?
I'm imagining an invigilator watching video feeds using an interface similar to, say, that used by security guards to monitor surveillance feeds. I would think that a single human invigilator can monitor more students using this system than an in-person setting. What am I missing?
On top of that, there's always a bunch of annoying mucking about getting set up for Zoom invigilation: angle of camera, light, checking environment, etc. All that needs communication to and fro, and it can take up to 5 minutes for a single student. Now multiply that by say 100 (a bit of parallelism is possible, but individual communication is needed with each student).
Given the cost of college courses, schools could easily afford $5 per student per course, and $100 for proctoring 20 students for a single exam is more than fair.
We have designed entirely new fully-online interactive courses in response to this.
In other courses, where we have e.g. 640 local students and 10 online, they need to sit the same test for it to be fair.
Just adjusting your lecturing to online can easily take an additional 2h preparation per lecture, and often takes significantly more, because you have to also adjust all communications and how to make material available. That's even without redesigning the course for online learning. I know a lot of colleagues who got very overworked, because of the increased demand on teachers during the pandemic.
So instead you'd rather waste money on a barely-functional piece of software that actively invades your students' privacy?
When that software is required to be installed on private machines and is difficult to remove, yes, it absolutely does make things worse.
I'm a student who takes special care about the software I install on my laptop. I use a Linux distro, run primarily open-source software, and sandbox every single proprietary app (limited access to files, no admin at all, no screen recording, disabled webcam, ...). I've also looked into several of these exam spyware tools (you really are forcing students to install spyware), and they're built with often hilariously poor security practices.
Which is to say nothing of the regularly stolen source code; If you held the exam spyware solutions to the same standards that you held students to, you would write up almost every single vendor to the Academic Integrity office. Another example of hypocrisy in academia from the perspective of a disgruntled student.
I deliberately do not install any video games with invasive anti-cheating functionality (and I regularly critique them, like I do for exam spyware); that is a false equivalence anyway, since they don't deal in the same breadth of personally identifiable information (like a permanently saved panorama of my bedroom).
Don't assume all students are the same.
Second, the consent dynamics are wildly different. For a game, its like "you trade this in for fun/relaxation" - and there's always other games that don't spy on you. I play those. With universities, many pulled a fast one and introduced the spyware to students after their tuition is already paid, and said "use it or drop the course". You can't switch universities because one university didn't consider the ethics of spyware; you can switch games much more easily.
That's why I find it so offensive that Proctorio is suing what appears to be university staff to silence them.
If you absolutely need this type of monitoring it should be done at some sort of testing center on cleanly imaged computers. I could see it being done for something absolutely crucial such as a bar exam.
Proctorio and its ilk should not be used for simple mid-terms or even finals. This monitoring software becomes a huge crutch to lazy administrators and teachers. I should know I am lazy too.
PS While we are on the slippery slope, if Proctorio becomes standard we rapidly head to Snow Crash situation where federal jobs were heavily monitored. Already many freelancer sites use screenrecording software.
TL;DR Canvas and Moodle use incrementing integers for both user ID and quiz ID. Proctorio's "zero-knowledge encryption" has a shared key derived from the two IDs; they store the user ID, so that's effectively a single PIN. With their older settings, you can brute force a quiz ID in a couple hours at most.
They increased the time cost for the brute force to now take days/weeks, but that's still peanuts and the attack scales really well, because most exams take place at the same time (students start/end at similar times), so once you crack the quiz ID for one record, that's tens-hundreds of records; and since IDs are just increasing numbers, once you find the lower bound, working your way upwards is much easier.
They also added an option for universities to use PGP keys - but that involves training faculty, or manual setup.
For more details, here's my blog article: https://proctor.ninja/wave-rake-proctorio
Why? How did we ever produce lawyers in the past without Proctorio?
I don't think it's a slippery slope. We will have significantly more local students than remote for the foreseeable future, and the local students are invigilated in person. We have students wanting to opt into remote study, including using this software, but so far if they're local we tell them no (except under extreme circumstances).
It is a huge slippery slope because this privacy invasive software ends up being used for such mundane things like weekly quizzes.
Do you really want to live in the future/present where we have to submit ourselves to daily monitoring?
How about adding some monitoring via sensors of performance boosting substances?
My answer is that such monitoring software should be only used under exceptional circumstances on neutral computers - my example was a testing center with insufficient human supervision.
This kind of software does not monitor computer use outside of the test. It monitors the student during the test, like a human invigilator (yes, it's recorded, so not identical). During the test, the student is only working on the test, which is not private or secret. When the test is finished, the software exits.
I don't see how you could have the resources for an infrequently used testing centre, but not enough to pay one person for an hour or two. That is not what this kind of software is useful for.
You fail to consider the circumstances in which the test takes place. Students take the test in their personal spaces, and earlier in the thread, you mentioned essentially inspecting a student's living space (...angle of camera, light, checking environment, etc...) "Checking environment" is really just a cold, "process" word for inspecting a student's living space.
A student's room can often have private or secret things about them. Before you ask, not every student has the privilege to use a separate, clean, blank room to take tests. A personal space is inevitably going to have personal, private things. I've brought this up before; I personally know friends who were outed to professors as trans because their personal space has things like needles - and then you even have stuff like naive professors assuming "drugs" when its really just medications.
It could be anything else besides that, in fact - calendars with things scribbled on them; family photos; posters for political organizations; if you look in someone's bedroom, you're inevitably going to find out things about them that they would rather you not know.
Would you take your students on a tour of your bedroom while you're teaching an online class?
EDIT: In addition, there's non-traditional students and high risk students, and interruptions in general - there's not _only_ a test going on - I've had someone from my family interrupted in the middle of an exam because someone from the government knocked the door to take our temperatures and ensure we're healthy and don't have COVID. There's always more things going on, too.
There are real concerns around privacy with software like this - what will happen if some criminal finds an exploit in Proctorio's software? This kind of monitoring software is often designed to be hard to detect or bypass, and installs itself in ways that are challenging to remove. Development of this kind of software should be done with a high degree of caution and care. There isn't a risk that an invigilator will be compromised by a malicious actor, and then see every detail of students lives and all their private data for all time. But this risk does exist for software.
If Proctorio takes legal action like this against critics what does that say about their software?
I agree about the privacy concerns, and in normal circumstances I wouldn't use proctoring software, because all my students would be local (or they would have signed up as distance students knowing this kind of thing would be needed). I would also have much less issue with it if it was open source.
I see it as a currently-necessary annoyance, as the least bad option. The alternatives have greater deficiencies: human invigilators using Zoom etc. don't scale (institution experience); oral exams don't scale (my experience moderating such assessments); no invigilation leads to cheating (by few students but enough to be a real problem, especially for professional qualifications - my direct experience and institution experience); shutting down education until the pandemic's over is unfeasible.
How is it a real problem (I've never cheated but always felt I got fair grades, not affected by whatever cheaters did)? If a student memorizes everything they need for the exam today, but forget all of it tomorrow, is that useful? That's a very real thing happening every day in exams, and nobody highlights it as as big a problem as cheating (I think it's a bigger problem actually, but the solution is not necessarily in finding those who can do that, but in optimizing the material so memorization is not such a core part of it before you start specializing).
In careers where such things really matter (think medicine), people only progress by demonstrating actual knowledge and understanding while gaining real-life experience. We don't get a surgeon out of a medical school that hasn't shadowed a surgeon and been quizzed on things to do next.
It's similar with engineering: you are given smaller, simpler things when you start off, and you build up your knowledge and experience before being given the role of a lead engineer for an airplane engine.
Exam grades are never used as a measure of someone's knowledge on the topic, though a paper they authored, or their dissertation, naturally, might.
Is it necessary though? Have you considered there are ways other than testing for a student to demonstrate their knowledge on a subject? Projects, presentations, and writing all come to mind as effective ways to measure knowledge on a subject and do not require treating all students like cheaters because a few choose to do so.
From talking to remote students, I don't think they feel like they're being treated like cheaters. Instead, they seem happy we're making their study possible, and accepting of what they're asked to do. They know it's important that they can demonstrate unequivocally that they have particular skills.
Even in early level mathematics, there are plenty of opportunities to introduce word problems that can only be solved by applying the relevant techniques. As long as the teachers are defining these word problems themselves (rather than pulling them from an online resource), they stand as a pretty good guard against cheating since they require students to first recognize the technique that needs to be applied, and then to extract the relevant variables from the word problem to apply that technique.
Furthermore, in early level mathematics, you can still have students present solutions to problems and explain why the solution works. For instance, say you were interested in whether or not a student has grasped the basics of derivatives - simply get on a call with that student, give them a random function to solve the derivative for, and then have them do so in front of you.
These are all things I've quickly thought of that would have at least be partially effective in measuring knowledge. I imagine any person with a career dedicated to instructing students could come up with many more options that could be even more effective.
> Instead, they seem happy we're making their study possible, and accepting of what they're asked to do. They know it's important that they can demonstrate unequivocally that they have particular skills.
Students are happy to be able to study and know that it's important to demonstrate their skills - but that doesn't mean that they wouldn't be happier if they could demonstrate their skills without the invasive testing software. I argue this is setting up a false choice: "you can either learn nothing at all, or do so under this cheating software". But the reality of the situation is that they can still learn and demonstrate their skills without the cheating software.
Not sure what your point is, but the difference between "dead president Hugo Chavez flipped votes for Biden" vs "Here is how the software works, I don't think it's right" is quite important.
> "suspicious" events get reviewed by a person
Being taped and having that tape scrutinized after the fact is not equivalent to being in the same room as an invigilator. I could accept a live video call as being roughly equivalent, but not if it’s recorded.
That's a fairly classist assumption that someone has multiple devices, including some older piece of junk that's late model enough to be useful for test taking but still essentially a "throw away." It implicitly means that adequate right to privacy is only reserved for wealthy kids and not for anyone who only has one computer.
That's not to suggest that I support this assumption even in cases where it's true. I'm just trying to point out a common blind spot that leads comfortably well-off people to often act with callous disregard towards those who have less because it isn't a big problem for themselves.
only downside is that it's a bit tricky to set up if you want to dual-boot two windows systems.
> In Linkletter’s view, customers and users were not getting the whole story. The software performed all kinds of invasive tracking, like watching for “abnormal” eye movements, head movements, and other behaviors branded suspicious by the company. The invasive tracking and filming were of great concern to Linkletter, who was worried about students being penalized academically on the basis of Proctorio’s analysis.
In an in-person invigilated test environment, the invigilator/proctor is watching students carefully for suspicious movements and behaviours. We don't call that an invasion of privacy. While I don't like commercial proctoring software (I have to use some, not Proctorio, for students who can't enter the country because of Covid-19), being videoed by a machine while you do a test is a) not much different from everyone else in the class who's being watched in person (you probably get watched less, actually), b) a way to make sure essential academic standards are upheld. No one is going to get penalised based on some fallible "AI" system; "suspicious" events get reviewed by a person and usually aren't suspicious at all.