News for hackers. Stuff that matters.
Re:Students think the College Board is running a Reddit sting by jackson1442 on Sunday May 17, 2020 @03:54PM (#23214827)
I took the AP Chem exam last Thursday, fortunately without incident. One of my friends was unable to submit his AP Literature assessment, facing the same error popup that has plagued so many of the other students. I've heard from several other students in my district that have had a similar problem. My brother had an AWS Lambda error message when he tried to access his exam the first time; CollegeBoard had surpassed their Lambda service limits (Fortunately he was able to access his exam after trying again).

Their online tooling has never been great–AP Classroom takes at least 3 tries for me to access, one to log in again which then redirects back to AP Central, one that just redirects back to AP Central, then finally one that takes me to my assignments page.

This certainly isn't an easy problem to solve. CollegeBoard definitely should have designed a more stable system to host their exams on, but as with anything digital, there's always an opportunity for error–even moreso since everyone takes the exam at the same time (which is an entirely different can of worms, with students in Guam taking exams at 1a, 3a, and 5a).

If I were to design these assessments, I'd have it be an untimed, synthesis-based assessment, more similar to AP Computer Science Principles[0], with their Digital Portfolio. Simply having a deadline for all students to upload to a basic tool (which already exists as CB's Digital Portfolio system). Like the current exam, have the submissions sent back to teachers for validation (there's nothing on the current system that validates your identity, simply paste in your AP ID and enter your name/dob, and you're in the system).

I think if a student can write a well-formed essay over a topic, they can get college credit. In science classes, they could design an experiment and simulate a lab report. CS classes can create a program, etc.

[0]: https://apcentral.collegeboard.org/courses/ap-computer-scien...

Re:Students think the College Board is running a Reddit sting by acbart on Sunday May 17, 2020 @04:25PM (#23215110)
I still have yet to hear anyone propose a workable solution for the ultimate problem of remote assessment in CS: hiring someone to just do your work for you. Everyone keeps saying "oh just make it project-based" like I didn't have a student last fall who hired someone to do his final project. I only detected him cause the person he hired sucked and left bread-crumbs; how many students did I not detect because they hired competent people?
Re:Students think the College Board is running a Reddit sting by whymauri on Sunday May 17, 2020 @05:04PM (#23215477)
Some companies, like Pinterest, use systems like "lytmus" to prevent cheating. It's a monitored VM box where you do the entire assignment inside the GUI VM. I'm sure certain trust factor guarantees prevent you from outsourcing the assignment overseas.

All that said, it's an awful system and I ragequit the take-home midway through. It's buggy, slow, lacks useful hotkeys, and basically requires you to become accustomed to a wiped Linux box without any of the tools you would typically use. Sure, you can install them - but it's a timed assessment, for crying out loud.

Finally, I suppose nothing stops you from logging in with your credentials on your authorized machine and then physically handing that machine to a paid agent.

Re:Students think the College Board is running a Reddit sting by erichurkman on Sunday May 17, 2020 @06:30PM (#23216103)
This is the same reason I offer candidates remote screen share-based technical assessments. Part of what makes an effective engineer is mastery over tools: by putting you into Coderpad or Hacker Rank type tools, I'm handicapping you.

Not everyone opts for that (maybe their local environment is messy, or they only have a locked down work device, etc), but I always make it an option.

I've learned many things over the years watching other experts in their local environments as an interviewer. New packages, new shortcuts, new helper apps, ...

Re:Students think the College Board is running a Reddit sting by neutronicus on Sunday May 17, 2020 @09:06PM (#23217115)
Ha.

I'm a Spacemacs user and I had to do the remote interview for my current gig in Google Docs. That was a bit frustrating.

Re:Students think the College Board is running a Reddit sting by derefr on Sunday May 17, 2020 @05:43PM (#23215771)
> I'm sure certain trust factor guarantees prevent you from outsourcing the assignment overseas.

Who said anything about overseas? Hire your classmate to sit down at your own PC to do it. That's the Hard Problem.

Re:Students think the College Board is running a Reddit sting by marcinzm on Sunday May 17, 2020 @06:26PM (#23216070)
I heard some companies, possibly only in India, required a webcam to be on the whole time and for you to rotate it around the room beforehand.
Re:Students think the College Board is running a Reddit sting by jwilliamson on Sunday May 17, 2020 @07:35PM (#23216556)
Wilfrid Laurier University in Canada is trying to make the students do this for exams this year.
Re:Students think the College Board is running a Reddit sting by noodlesUK on Sunday May 17, 2020 @07:06PM (#23216367)
Yeah and this stops me pre-recording it how? I have a USB capture card that appears to the host machine just as a UVC webcam.
Re:Students think the College Board is running a Reddit sting by zeta0134 on Sunday May 17, 2020 @09:08PM (#23217134)
When I took my RedHat RHCSA exam, the proctor was a live human, and I needed to move the webcam in time with their instructions. It was a basic around-the-room pass, but it covered anywhere that I could have stashed materials with which to cheat. From there, even if I did have external materials, it would have been pretty obvious that I was using them because my eyes would be focused offscreen for an extended period of time.

Just the knowledge that another human being was proctoring the exam made the idea of attempting to cheat pretty far fetched. My body language would have given it away, and it wouldn't be easy to transition smoothly from the interactive setup to some kind of pre-recorded loop without it being easily detected. The kind of person who could successfully pull that off... well, could also probably pass the test on their own merits.

Re:Students think the College Board is running a Reddit sting by StillBored on Monday May 18, 2020 @12:50AM (#23218481)
I suspect people are making this more complex than it needs to be. For most of these tests the difference between a good score and a poor one are just a few questions.

So, the first step is just to actually take the test. The cheating bit would be having someone monitoring what your doing via a reasonably hi-res hidden camera/etc pointed at your screen/etc. Then all you need is a channel for that person to communicate with you. Be that a tiny bluetooth hearing aid in one ear, a phone/screen taped hidden on your leg/etc or to your existing screen. Basically somewhere not visible to any camera's monitoring you.

So, when the person monitoring you detects you have answered something incorrectly they send you something to the effect "question #5's correct answer is Y because A,B,C". Then later you get a moment of insight and go back and fix the problem. In theory, unless your completely clueless the difference between a 2 and a 4 is just a couple questions so this would only have to happen a couple times during the whole test. Again, unless your completely clueless, if the proctor notices you fixing an incorrect answer and asks "why did you just change your answer" you give them something reasonable sounding. Heck for some of them, it might not even have to be the right answer, its completely possible for someone to get the right answer the wrong way in a multiple choice section. AKA, "I guessed on that one the first time and A & B seemed far fetched so I just picked C over D, but then later I questioned my assumption and decided it was B & C that seemed far fetched so I picked the remaining answer which was D rather than A".

Anyway, I'm not sure the screen is needed. For a test that is 100% multiple choice, a buzzer in your shoe would be enough if it went off whenever you selected a wrong answer, and then pulsed a code to indicate the right answer.

Re:Students think the College Board is running a Reddit sting by elliekelly on Sunday May 17, 2020 @09:34PM (#23217265)
Did they ask for a photo of your laptop? Particularly behind the lid? That’s the one place your webcam can’t see so it seems like it would be the ideal spot to hide or mount a cheat sheet.
Re:Students think the College Board is running a Reddit sting by zeta0134 on Monday May 18, 2020 @01:56AM (#23218862)
Ah, no but the exam was taken at a kiosk, not on my personal machine. I had only a few minutes in the room to get myself situated. The camera was USB, so I moved that around while the laptop remained stationary.
Re:Students think the College Board is running a Reddit sting by markrages on Sunday May 17, 2020 @11:59PM (#23218176)
Could you have hidden a cable to another monitor/keyboard in another room?
Re:Students think the College Board is running a Reddit sting by 9nGQluzmnq3M on Sunday May 17, 2020 @07:51PM (#23216662)
Ask a few questions, including at least one that's unpredictable.
Re:Students think the College Board is running a Reddit sting by ceejayoz on Sunday May 17, 2020 @09:08PM (#23217132)
"Touch your left cheek. Now show me the ceiling. Smile at the camera. Blink three times quickly."

If you can pre-record stuff like that without knowing what the sequence of requested events will be, I'm impressed.

Re:Students think the College Board is running a Reddit sting by renewiltord on Monday May 18, 2020 @02:20AM (#23218997)
Oooh, it would be cool to deepfake oneself on this. On an intentionally downsampled video feed it could be quite convincing.
Re:Students think the College Board is running a Reddit sting by saagarjha on Monday May 18, 2020 @03:30AM (#23219382)
While I would agree that the environment is torture to use as a general-purpose screening tool, I’m sure there are jobs where the ability to get work done in a base Linux install is an asset :)
Re:Students think the College Board is running a Reddit sting by pmiller2 on Sunday May 17, 2020 @07:18PM (#23216436)
Wow, when did Pinterest implement that? I don't remember anything like that when I interviewed with them, but it was a couple years ago.
Re:Students think the College Board is running a Reddit sting by whymauri on Sunday May 17, 2020 @08:28PM (#23216892)
2017 through early 2018. I think (and hope) that they've changed the process.
Re:Students think the College Board is running a Reddit sting by pmiller2 on Sunday May 17, 2020 @08:30PM (#23216905)
Ah, okay, it was late 2018 when I interviewed there. I guess they got rid of it since.
Re:Students think the College Board is running a Reddit sting by letitbeirie on Monday May 18, 2020 @01:29AM (#23218717)
When your students submit the project, have the landing page give them 10-15 minutes to make one tiny change and resubmit. Pick something that would be trivial to do in seconds for a student who did his own work, but would cause a lot of stress for someone who outsourced.

Save the 1:1 interviews for students whose original program was great but whose modification went down in flames and you'll likely be talking to your outsourcers, plus identifying students who maybe could use some extra help even if they did everything themselves.

Re:Students think the College Board is running a Reddit sting by wdevanny on Monday May 18, 2020 @02:54AM (#23219173)
Are you prepared on the basis of a student not successfully making the change to fail that student and/or expel them from their university? Sure you can invite them in for an interview, but what happens if the student just denies any accusations.

I am also skeptical you could find appropriate trivial changes to request. Students, and particularly beginner students, may write very unconventional code that may not be as easily modified as a well-thought-out solution.

Re:Students think the College Board is running a Reddit sting by jdsully on Monday May 18, 2020 @06:23AM (#23220584)
If its something like modifying a string literal then I can’t see the problem. If the program is too obtuse to permit that then its a major concern on its own.
Re:Students think the College Board is running a Reddit sting by jgwil2 on Monday May 18, 2020 @11:10AM (#23223304)
If it's just modifying a string literal I suspect most would just search for that string and have little trouble even if the work is outsourced. Perhaps within a certain set of requirements there are modifications you could ask for that are not trivial and could be done in fifteen minutes but I think this is pretty hard in the general case and depends a lot on code quality.
Re:Students think the College Board is running a Reddit sting by saagarjha on Monday May 18, 2020 @03:27AM (#23219366)
I guess you could take those students and review their cases personally by taking to them.
Re:Students think the College Board is running a Reddit sting by true_religion on Monday May 18, 2020 @02:50AM (#23219151)
This will work for one semester. The next semester everyone will know about it, and keep their outsources on staff till after grades are given.
Re:Students think the College Board is running a Reddit sting by chrisseaton on Sunday May 17, 2020 @04:45PM (#23215292)
> I still have yet to hear anyone propose a workable solution for the ultimate problem of remote assessment in CS

What about 1-on-1 interviews? Ask people the questions face-to-face, ask them to talk you through their answers. That's how advanced degrees like PhDs are assessed.

Re:Students think the College Board is running a Reddit sting by indigochill on Sunday May 17, 2020 @05:06PM (#23215492)
This is also what my team does for professional interviews. We give them a take-home test with some simple problems, but we actually care much more about how they talk through their solution with us than the solution itself. We don't even care if they copy/pasted stuff from Stack Overflow.

What we care about is that they can demonstrate they understand the problem and solution. The code is just the starting point for that conversation.

We have hired people who submitted failing solutions because they were able to think through the problem on the spot when we told them it failed and asked why.

Re:Students think the College Board is running a Reddit sting by acbart on Sunday May 17, 2020 @05:48PM (#23215806)
I'm 1-on-1 interviewing my 59 Algorithms students this semester. 20 minute interviews, and I have 2.5 TAs to help offset the workload. Starts on Wednesday, I'm absolutely dreading it. This would never scale to my 180 Intro students that I'll be getting this fall.
Re:Students think the College Board is running a Reddit sting by lonelappde on Sunday May 17, 2020 @08:08PM (#23216767)
Recruit proctors from big companies.
Re:Students think the College Board is running a Reddit sting by turndown on Sunday May 17, 2020 @06:17PM (#23216004)
What are the 59 algorithms?
Re:Students think the College Board is running a Reddit sting by chrisseaton on Sunday May 17, 2020 @06:18PM (#23216011)
59 students of algorithms, not students of 59 algorithms, I would guess.
Re:Students think the College Board is running a Reddit sting by chongli on Sunday May 17, 2020 @09:08PM (#23217137)
I would definitely take a course entitled "59 Algorithms". What a fantastic idea for a second course in algorithms! Looking at 59 of the most important algorithms (in the opinion of the professor). It's even more intriguing since it's not a round number, so you know the prof didn't add a bunch of filler just to get a nice number.
Re:Students think the College Board is running a Reddit sting by ahefner on Sunday May 17, 2020 @11:12PM (#23217922)
"59 Algorithms" would be a fantastic book title. I'd buy it.
Re:Students think the College Board is running a Reddit sting by acbart on Sunday May 17, 2020 @08:22PM (#23216844)
Yes, thank you for the clarification :)
Re:Students think the College Board is running a Reddit sting by tnecniv on Monday May 18, 2020 @02:10AM (#23218932)
In my experience, this is definitely the best way to evaluate how competent someone is in an area. If they can easily hold a casual conversation and express / defend some sensible opinions about an area, it's likely they know what they're talking about. Obviously, you should ask technical questions in the interview, but it's the little things in the conversation that say a lot. Unfortunately, this approach doesn't scale to the size of the AP Exams.

Dijkstra exclusively gave oral final exams on a board. Allegedly, he stopped one of his former students after the first question and told him something to the effect of "you clearly know what you are doing by how you answered that, so you'll get an A, but your handwriting is atrocious." He spent the rest of the exam time making the student practice penmanship.

Re:Students think the College Board is running a Reddit sting by acbart on Tuesday May 19, 2020 @03:44PM (#23239261)
Djikstra - not a professor I would want to emulate.

I'm unconvinced that oral exams provide the most secure and consistent form of assessment. I'm not even sure how to evaluate such a claim, though, so it may have to stand as an opinion.

Re:Students think the College Board is running a Reddit sting by im3w1l on Sunday May 17, 2020 @06:11PM (#23215965)
There are entertainers streaming with appearance-modifier and voice-modifier filters. You can't know you are interviewing the person you think you are.

I don't think there is an off-the-shelf product for cheaters, but it's only a question of time. Especially now that I mentioned it.

Re:Students think the College Board is running a Reddit sting by chrisseaton on Sunday May 17, 2020 @06:36PM (#23216150)
I think ideally you should know your students well enough to be able to judge whether they're giving you answers which correspond their ability and their own take on the topic. If you've been tutoring them up to this point in the year you should know them pretty well.
Re:Students think the College Board is running a Reddit sting by dahfizz on Sunday May 17, 2020 @09:59PM (#23217405)
I don't know if you have been to college recently, but I had a handful of intro classes with 200+ students. I never spoke to those professors once.
Re:Students think the College Board is running a Reddit sting by jancsika on Monday May 18, 2020 @12:54AM (#23218502)
Those aren't classes. They are class theater.

They probably failed even by the simple arithmetic that the time required to make a minimally good faith effort at applying a grading rubric is greater than the time a teaching assistant is paid to "assist" in the grading.

Your teaching assistants almost certainly scaled back their time grading your assignments to fit their allotted hours. If any of those intro classes fulfilled a writing credit you can probably start with available undergrad writing credit requirements, figure out how many hours your TA got paid, and look back at the number of students for a given TA to figure out just how much money you and/or your parents lost on the deal.

Re:Students think the College Board is running a Reddit sting by dahfizz on Monday May 18, 2020 @11:46AM (#23223885)
> figure out just how much money you and/or your parents lost on the deal.

If you consider college tuition to be the money you pay to be educated, you're definitely right. Personally I think of college as something like a licensing or certification.

I learned twice as much at my part time programming job than I did in class (but classes where instrumental in getting me through technical interviews). But, once I had my degree I instantly had multiple 100k+/yr offers in a moderate COL area. For me, that is a good deal (but still a huge hassle).

Re:Students think the College Board is running a Reddit sting by jancsika on Tuesday May 19, 2020 @04:49PM (#23240009)
> If you consider college tuition to be the money you pay to be educated, you're definitely right.

If I think of class theater in an education that led me to 100k+/yr, that is a good deal.

If I think of class theater in a certification that led me to 100k+/yr, that is a good deal.

That it is a good deal doesn't change the the fact that class theater is a shady practice.

It's also detrimental. But since we don't have the tech to do speed runs against the versions of ourselves who took non-shady intro classes it can be easy to shrug.

Re:Students think the College Board is running a Reddit sting by acbart on Tuesday May 19, 2020 @03:54PM (#23239376)
Sadly, certification is more or less our job now in higher education. Wasn't how the system was designed, and isn't what most faculty think about. But it's really becoming accurate.

And I can't do that without proper assessments!

Re:Students think the College Board is running a Reddit sting by noodlesUK on Sunday May 17, 2020 @07:08PM (#23216378)
Yes, you would hope so in smaller, more specialised classes. However, in a ~200+ person class, you just won’t have that kind of relationship with each student. Additionally, this completely breaks the anonymous marking that many institutions strive for.
Re:Students think the College Board is running a Reddit sting by NullPrefix on Sunday May 17, 2020 @05:03PM (#23215470)
1on1's take too long and don't InternetScale^tm
Re:Students think the College Board is running a Reddit sting by dasudasu on Sunday May 17, 2020 @09:10PM (#23217144)
This is how all exams at the university level used to be conducted. Not really practical however when you have large class sizes.
Re:Students think the College Board is running a Reddit sting by jackson1442 on Sunday May 17, 2020 @05:12PM (#23215540)
It's definitely a problem, even on CollegeBoard's current platform. Normal AP exams/SATs require photo ID validation at the exam site, and are run by school admins who will notice if someone is out of place. I work as a tutor on Wyzant and have been asked to "tutor" during the exam as well as to just do CS Principles students' portfolio assignments.

While some might argue my approach is too aggressive, I report every account that messages me asking to cheat to Wyzant as well as the appropriate organization. My goal is to make it simply not worth the attempt to cheat (obviously only if they explicitly ask; I don't report when it's a possible miscommunication). Just about all of them have had their accounts deactivated.

If technology access could be assumed, I don't think it would be unreasonable to ask students to upload a picture of themselves with their photo ID, as well as some sort of validity check, like a unique code only provided to the device they're taking the exam on. Obviously, that can't be implemented because of equity issues. Ideally, a secured assessment would be done on something similar to Pearson Vue, ProctorU, or the multitude of other online proctoring services. Unfortunately, that's not an option due to both scale and technology access (not everyone has a compatible device, I'm sure many students are taking the exam on their phones).

Re:Students think the College Board is running a Reddit sting by GhostVII on Sunday May 17, 2020 @04:31PM (#23215168)
You could make them give a video presentation about their project to show that they actually know how it works, not foolproof but it helps. I think it is pretty much impossible to completely remove that possibility though. I've had lots of courses where the majority of my mark is assignments and projects, and I'm sure some students pay others to do it for them, and the professors just accept that as a possibility. I don't think it is worth providing a worse experience to the majority of honest students to prevent a small minority of dishonest students from cheating.
Re:Students think the College Board is running a Reddit sting by elliekelly on Sunday May 17, 2020 @05:53PM (#23215849)
Do you have them upload the source code of their projects? If so, after the upload could you display the project’s file tree along with a last minute change their hypothetical client has requested and give them a minute or two to select the file(s) that would need a pull request in order to make the change? They don’t even need to make the change, just identify where the change would happen.

Someone who wrote the code themselves will know right away and someone who purchased a project won’t have enough time to sift through the code to figure out the answer.

Re:Students think the College Board is running a Reddit sting by acbart on Sunday May 17, 2020 @08:24PM (#23216856)
Well, it's not an easy suggestion, but certainly an interesting one. I'm going to think more about this one. Thanks for providing a new thought on a hard problem!
Re:Students think the College Board is running a Reddit sting by elliekelly on Sunday May 17, 2020 @10:00PM (#23217412)
The problem reminds me of that old story about the group of kids who missed their final exam with a flat tire. If you can figure out the modern CS equivalent(s) of asking your students “which tire” about their project there’s probably an easier implementation. Good luck!
Re:Students think the College Board is running a Reddit sting by vulcan01 on Sunday May 17, 2020 @04:43PM (#23215278)
An interesting way to assess your students, if you have time and a reasonable amount of students, is to do what the IB organisation required my Spanish teacher to do: a one-on-one oral exam with presentation and follow-up questions from the teacher, all recorded and sent to the IBO grading board.
Re:Students think the College Board is running a Reddit sting by acbart on Sunday May 17, 2020 @05:49PM (#23215810)
I'm 1-on-1 interviewing my 59 Algorithms students this semester. 20 minute interviews, and I have 2.5 TAs to help offset the workload. Starts on Wednesday, I'm absolutely dreading it. This would never scale to my 180 Intro students that I'll be getting this fall.
Re:Students think the College Board is running a Reddit sting by wombatpm on Sunday May 17, 2020 @08:49PM (#23217021)
But doesn't the whole 101 CS classes really not scale in general. I remember having code submission submitted to to some anti-plagiarism system back in the 90's.
Re:Students think the College Board is running a Reddit sting by acbart on Tuesday May 19, 2020 @03:57PM (#23239403)
I like to the curriculum I'd been developing for the past few years was scaling pretty well. Most of the technology is reaching enough stability that I can give a ton of automated feedback without killing myself, I had a pretty low DFW rate, most students could complete my final programming problems in a proctored exam situation, and evaluations came back high. I wasn't happy with my project rubric scores, but I think I was on the way to progress with that and it was partially just too high expectations (I just want students to test and decompose more). I'm interested in how we can scale introductory computing experiences, and I think I was on a good path before they yanked out my proctored exams out from under me :)
Re:Students think the College Board is running a Reddit sting by dntbnmpls on Monday May 18, 2020 @02:43AM (#23219117)
The only way is in-person student code review or in-class projects. You could go over their code with each student and ask poignant questions that can only be answered if the student wrote the code themselves. But I'm not sure how feasible that or the in-class project is for most teachers.

Also, anyone remember the story of the programmer who subcontracted his work to some offshore freelancer for a fraction of his salary? He just sat at his cubicle goofing off all day while the freelancer did all the work.

Re:Students think the College Board is running a Reddit sting by Frondo on Sunday May 17, 2020 @04:29PM (#23215150)
Could something like pair programming work to reduce the risk of hiring out the whole project?
Re:Students think the College Board is running a Reddit sting by acbart on Sunday May 17, 2020 @05:49PM (#23215817)
Reduces the risk, but doesn't prevent it.
Re:Students think the College Board is running a Reddit sting by entee on Sunday May 17, 2020 @04:22PM (#23215081)
I agree to a certain extent on your revised assessment framework, but I’m not sure it would work very well in some of the basic sciences. As someone who has gone to college for these things and beyond, I can tell you there’s very little as a “practicum” that I could have demonstrated coming out of high school or early college. This is in contrast to CS, where if you build a program, the fact that the program exists and solves a given problem correctly is ipso facto a demonstration of understanding. Further, it’s a demonstration of fairly comprehensive understanding of the coursework (you’d have to grasp >80% to have the program work at all), which I have trouble seeing how to do that in say, chemistry.

You propose a lab report, with example data, and I actually really like that idea, but I think it would fail on that last score. Looking back, AP chem is a lot of inorganic reactions (lots of memorization, some principles), acid base work, some organic chem, some lab techniques. It’s fairly scattered. A single (or several) lab reports would not capture all of it, especially given that chemistry doesn’t work perfectly predictably even with a post-graduate understanding. There’s a reason basic science lab courses do boring experiments, it’s because those work reliably.

Anyway, I like the idea, and want to hear more on how to do it, but a lab report would have to be supplemented by something else to really evaluate understanding in the sciences.

Re:Students think the College Board is running a Reddit sting by jackson1442 on Sunday May 17, 2020 @05:24PM (#23215619)
AP Chem is definitely an exercise in memorization and general "knowledge of chemistry" for a good portion of the class. Though not a complete solution, a lab report-based assessment could look more similar to the CS Principles exam in another facet: written response.

On the APCSP exam, there's actually very few points for your code itself. You get a point for identifying an abstraction and a point for identifying an algorithm, but the rest of the points (there are 8 total) are assigned based on your written responses. Analytical questions could be added to each exam–broad enough that no two students should have too much overlap, but narrow enough that it's clear what is expected of a student response.

From there, a few points could be assigned for the quality of the designed experiment, but a majority would be assigned based on the quality of the written response questions.

AP Physics follows a similar model on the regular paper exam–you have to prove some theory and design a complete experiment to do so. I believe it also included one light analysis question.

Re:Students think the College Board is running a Reddit sting by derefr on Sunday May 17, 2020 @05:49PM (#23215820)
> Further, it’s a demonstration of fairly comprehensive understanding of the coursework (you’d have to grasp >80% to have the program work at all), which I have trouble seeing how to do that in say, chemistry.

Lock someone in a room with some reagents, and expect them to produce extracted/cleaned output products of a certain testable purity and mass after N time. Make the process of getting from the reagents to the final product really involved, requiring at least one of every kind of basic reaction/setup taught in the course. (NileRed's "aspirin to tylenol" sequence might be a good example challenge: https://www.youtube.com/watch?v=b0Ejuew2riA)

You know this is "the practical approach", because this is essentially how gangs qualify their chemists. "If you can really make [illegal drug], then make us some! Without doing anything that'll get the police called! In four hours!"

Re:Students think the College Board is running a Reddit sting by slavik81 on Sunday May 17, 2020 @11:10PM (#23217907)
I had excellent knowledge of Chemistry theory, but I could not for the life of me get the expected results in lab. Like, I know that you should not be able to make a battery out of a lead anode and a lead cathode in distilled water, but I did.

That sort of practicum may be a reasonable test, but could give very different results than the normal written exam.

Re:Students think the College Board is running a Reddit sting by LeifCarrotson on Sunday May 17, 2020 @06:46PM (#23216232)
The entire problem seems to be that the College Board (and ACT and SAT) took an initial design that works fine in a proctored classroom and applied it to a global testing scenario where it doesn't fit.

> If I were to design these assessments, I'd have it be an untimed, synthesis-based assessment...

Yes, but that takes more resources and has more variability among reviewers. It does have the advantages of being more fair for students with time zone issues or closures, and of being durable in the face of temporary website issues. Judging by their decisions, the priorities of the testing agencies seem to have been taking out the cost of the human element of reviewers (multiple choice is easy to grade, synthesis essays need expert reviewers) and the variability of the tests and reviewers to certify their results for schools.

Re:Students think the College Board is running a Reddit sting by Gaelan on Sunday May 17, 2020 @06:50PM (#23216259)
Normal AP tests are half multiple choice and half "free response." What "free response" means depends heavily on the test, but they are always human-graded. This year, the tests are free-response only.
Re:Students think the College Board is running a Reddit sting by pirocks on Sunday May 17, 2020 @06:58PM (#23216312)
Human graded is a generous way of describing what they do with free response. When I took ap tests a large part of the preparation was knowing exactly how they where going to grade the questions, and optimising answers to hit as many points on the answer key.
Re:Students think the College Board is running a Reddit sting by bsder on Sunday May 17, 2020 @11:43PM (#23218094)
Yeah, "free-response" is only true if you're unprepared.

A significant part of taking AP English when I went through was the teacher saying: "Okay, here is the question. Now, what are the likely literature references the AP reader expects you to use? Okay, what are the likely scoring points the AP reader has on their worksheet? Okay, now write to those. Writing anything else is a waste of time."

You could count the number of students out of her class who didn't score a 5 on a single hand.

Re:Students think the College Board is running a Reddit sting by cxseven on Monday May 18, 2020 @02:18AM (#23218984)
Wow, none of my AP teachers back in '97 offered anything close to this advice. I wonder if that was normal for the time, or if I just experienced the low end of the variance of preparation.
Re:Students think the College Board is running a Reddit sting by bsder on Monday May 18, 2020 @05:20PM (#23228509)
Possibly we could be considered the high-end of preparation.

The English teacher in question was an actual AP Reader as well as an assistant editor of the local newspaper.

Re:Students think the College Board is running a Reddit sting by akhilcacharya on Monday May 18, 2020 @06:14AM (#23220527)
> My brother had an AWS Lambda error message when he tried to access his exam the first time; CollegeBoard had surpassed their Lambda service limits (Fortunately he was able to access his exam after trying again).

This is absolutely hilarious. Did they not realize the concurrency limitations?

Re:Students think the College Board is running a Reddit sting by joncrane on Sunday May 17, 2020 @05:02PM (#23215463)
I wouldn't be shocked if College Board just signed up AWS ProServe to a big contract.

If not, details in profile, College Board!