Clearly, the person put 0 effort towards cheating (as most cheaters would, to be fair). But slightly adjusting the prompt, or just paraphrasing what ChatGPT is saying, would make the issue much harder to spot.
I’m not sure why anyone would want a job they clearly aren’t qualified for.
$$$,$$$
It's a tool, and if they can master it to make it useful, then credit to them.
Alas, ChatGPT seems to be a jack of all trades, but master of none, which is gonna make it hard to pass my interviews which test very specific technical skills.
Tool usage is what separates us from animals and is generally ok where tools are available/expected, but in this case I think you misunderstand which tool we're talking about. The tool involved isn't actually chatGPT, it's more like strategic deception. Consider the structurally similar remark "as a voter, if a candidate can use lies to represent themselves as better than other candidates, I'm not gonna mark them down for use of dishonesty".
The rest of this comment is not directed at you personally at all, but the number of folks in this thread who are extremely eager to make various excuses for dishonesty surprised me. The best one is "if dishonesty works, blame the interviewer". I get the superficial justification here like "should have asked better questions", but OTOH we all want fairly short interview processes, no homework, job-related questions without weird data-structures and algorithms pop-quizes, etc, so what's with the double standards? Hiring/firing is expensive, time-consuming, and tedious, and interviewing is also tedious. No one likes picking up the slack for fake coworkers. No one likes being lied to.
Not me. I see it all the time, online and offline. I suspect they think it confers status on themselves, but what actually happens is honest people wind up shunning them.
If someone aces the interview using an LLM and then does good work using that same LLM then what should the employer or other employees care? The work is getting done, so what's the problem?
Compare a shitty worker to a deceptive one using an LLM. They both passed the interview and in both cases the work isn't being done. How are those two cases different?
I'm doubting this quite a bit. If it's so easy to ace an interview, why are there so many bad ones?
>Anyone can ace and interview and then slack off once they have to job.
In that a person can pass an interview, get hired, and then not do the job. An interview will never tell you if you will get poor job performance with 100% accuracy.
Interviews aren't perfect, but they can still be good filters.
Maybe I'm wrong, but I find it very hard to believe that anyone thinks the "good work" part here is actually a practical possibility today. Boilerplate generation is fine and certainly possible, and I'm not saying the future won't bring more possibilities. But realistically anyone that is leaning on an LLM more than a little bit for real work today is probably going to commit garbage code that someone else has to find and fix. It's good enough to look like legitimate effort/solutions at first glance, but in the best case it has the effect of tying up actual good faith effort in long code reviews, and turns previously productive and creative individual contributors into full-time teachers or proof-readers. Worst case it slips by and crashes production, or the "peers" of juniors-in-disguise get disgusted with all the hand-holding and just let them break stuff. Or the real contributors quit, and now you have more interviews where you're hoping to not let more fakers slide by.
It's not hard to understand that this is all basically just lies (misrepresented expertise) followed by theft. Theft of both time & cash from coworkers and employers.
It's also theft of confidence and goodwill that affects everyone. If we double the number of engineers because expectations of engineer quality is getting pushed way down, the LLM-fakers won't get to keep enjoying the same salary they scammed their way into for very long. And if they actually learn to code better, their improved skills will be drowned out by other fakers! If we as an industry don't want homework, 15 interviews per job, strong insistence on FOSS portfolio, lowered wages, and lowered quality of life at work.. low-effort DDoS both in interviews or in code-reviews should concern everyone.
You found a person (+ tool combo) that can do the job. If that person (+ tool combo) then proceeds to do the job adequately, is there a problem?
If you present a scenario in which a person passes the interview and then doesn't do the job, the you are answering a question I didn't ask.
To you scenario I would respond: the interview wasn't good enough to do its job, the whole point of the interview process is to find people (+ tool combos, if you allow) that can do the job.
>what's the problem?
Using a LLM is akin to copy/pasting code from random places. Sure, copy/paste can be done productively, except ChatGPT output comes completely untested and unseen by intelligent eyes. There are also unsolved copyright infringement issues via training data, and a question as to whether the generated code is even copyrightable as it is the output of a machine.
Find someone with a great resume and horrible interview skills. Chances are they have been working for years and are entering the job market for the first time. You are one of the firsts in their interview process. Grab them right away because once they start getting slightly good in the interview process someone will snap them up and realize they got a 10x (whatever it means to that company).
You'll never find that 10x if you are looking at interview performance unless you can compete on price and reputation.
Interview skill is not some monotonically increasing quantity. It very much depends on how the question hits you and what kind of a day you've had. Also, it somewhat depends on the interviewers' subjective interpretation of what you do. If you're more clever than them, your answer may go over their head and be considered wrong. They might also ask a faulty question and insist it is correct.
I'm not great at interviews myself. My resume is decent, but the big jobs usually boil down to some bs interviews that seem unnecessarily difficult to pass. I don't practice much for them, because I feel like it mostly depends on whether I've answered a similar question before and how I feel that day. I also often get a good start and just run out of time. I've found that sometimes interviews are super hard when the interviewers have written you off, as in you presented poorly in an earlier session and they are done with you. Also, when there is zero intention of hiring you generally, like someone else already got the job in their minds.
It does not. Please ask any LLM for examples of animals that use tools. (My examples: chimpanzees, gorillas, elephants, dolphins, otters...)
At my company we tell people that they should feel free to google or consult references at practical coding challenges.
> It seems hard to test someone's knowledge that way.
I don’t really want to test knowledge but skill. Can you do the thing? At work you will have access to these references so why not during the interview?
Now that doesn’t mean that we are not taking note when you go searching and what you go searching for.
If you told us that you spent the last 8 years of your life working with python and you totally blank on the syntax of how to write a class that is suspicious. If you don’t remember the argument order of some obscure method? Who cares. If you worked in so many languages that you don’t remember if the Lock class in this particular one is reentrant or not and have to look it up? You might even get “bonus points” for saying something like that because it demonstrates a broad interest and attention to detail. (Assuming that using a Lock is reasonable in the situation and so on of course :))
I do want to understand their knowledge. I'll preface questions with the disclaimer that I am not looking for the book definition of a concept, but to understand if the candidate understands the topic and to what depth. I'll often tell them that if they dont know, just say so. I'll start with a simple question and keep digging deeper until either they bottom out or I do.
It's like in college when you're allowed to take textbooks to an exam. You can bet the professor spent more time crafting questions that you can't answer blindly.
That being said, I think both types of questions have their place in an interview process. You can start with the no searching allowed questions in the beginning to assess real basic knowledge and, once you determine the candidate has some knowledge, you start probing more to see if they can connect the dots, maybe it's architecture decisions and their consequences, maybe it's an unexpected requirement and how they would react, etc.
What happened to all those folks? They retired, and turned into Boomers who are now unable to function in society at a basic level and do things like online banking or operate a smartphone.
You're on your own matey.
But hey, you'll have Google.
Even if you don't mind that situation, shouldn't you get buddy's contact information and offer him the job?
A better analogy is an interview where you can use a calculator (and not be detected). If the interviewer were only to ask you simple arithmetic questions with numeric answers then sure you'd seem to do well. So interviewers adjust to not doing that.
I mean, why not? Me and my buddy are a team, hire us both or none of us. Split the salary if you must.
Yeah, hiring someone to code a website isn't the same as maintaining a nuclear plant, but it's the same concept of someone that knows their craft vs. someone that needs to rely on tools. There's a major difference in my mind.
Similar it is unreasonable and bordering on negligence to assume a person has the skill set unique to your situation.
I can see how an applicant who cheats interview with chatbot would later not bother to internalize operation instructions for the job.
My experience with Chat GPT ranges from “it’s really good for rapidly getting a bearing with a certain topic” to “it’s a woeful substitute for independently developing a nuanced understanding of a given topic.” It tends to do an OK with programming and a very poor job with critical theory.
Exactly. It “only” shows you can & willing to at least understand the requirements, internalize them well enough, and comply with them. It shows your capability of understanding & working together with other humans.
Which is key.
In my impression, almost always the knowledge you receive at the uni is not really pertinent to any actual job, and anyone can have PhD level understanding of a subject without having finished high school.
It is the capability of understanding and working in a system that matters.
Similarly with a chatbot. Using it to game interviews in ways described does not mean candidate is stupid, or something like that. It is, though, a negative signal of one’s willingness and intrinsic motivation to do things like internalizing job responsibilities & procedures, or just simply behave in good faith.
Mental capacity to do mundane things is often important when it comes to, say, maintaining a nuclear reactor.
> just a tool
> it’s really good for rapidly getting a bearing with a certain topic
Perhaps. Personally I prefer using Google, so that I at least know who wrote what and why rather than completely outsourcing this to an anonymous team of data engineers at ClosedAI or whatnot, but if it is efficient to get some knowledge then why not?
It’s using it to blatantly cheat and do the key part for you where it becomes questionable.
Normally people who get bad results from it would also get similar results if they asked a domain expert. Similarly different knowledge domains use a different corpus of text for their core axioms/premises, so if you don't know the domain area or those keywords your not going to be able to prime the model to get anything meaningful from it.
Consider e.g. being a pilot, or a surgeon - two other occupations known for their extensive use of operational procedures today. People in those jobs are not being hired for their ability to stick to a checklist, but rather for their ability to understand reasons behind it, and function without it. I.e. the procedures are an important operational aid, not the driver.
Contrast with stereotypical bureaucrats who only follow procedures and get confused if asked something not covered by them.
Now, IMHO, the problem here is that, if you're hiring someone who relies on an LLM to function, you're effectively employing that LLM, with its limitations and patterns of behavior. As an employer, you're entitled to at least being made aware of that, as it's you who bears responsibility and liability for fuckups of your hires.
to your point though, a person needs both. all of one and none of the other is useless. You don't want someone who doesn't know what they're doing to play around disabling safety systems so you don't get Chernobyl, but for the everyday crud website you can just hire the coding monkey at a reduced cost.
I think that makes you an incompetent interviewer, unless your questions are too hard for ChatGPT. In any case, solving the question without ChatGPT is more impressive than using it. Just like most other tools, like search engines or IDEs.
Well, I suck at interviewing and/or leetcode questions, but have so far done perfectly fine in any actual position.
I can totally see how you’d resort to ChatGPT to give the interviewers their desired robotic answers after 3 months of failing to pass an interview the conventional way.
And yes, most large companies have terrible interviewers.
I've never once seen an interviewer getting better in any company I've worked for. What happens is they just move onto the next interviewee.
As someone who has interviewed a lot of people – robotic answers are specifically not what I (we?) look for. The difference between hands-on experience and book knowledge is exactly what we're trying to tease out.
It's very obvious when someone is reciting answers from a book or google or youtube or whatever vs. when they have actually done the thing before.
For the record: ChatGPT is very good and the answers it gives are exactly the kind of answers that people with book knowledge would give. High level, directionally correct, soft on specifics.
I mostly interview seniors, you obviously wouldn't expect experience from an entry-level candidate. Those interviews are different.
Here's Pew's Janna Anderson in 2015:
"Algorithms are taking over much of the human work of hiring humans. And, unless they are programmed to seek out currently undervalued and difficult-to-track factors, they may tend to find that the more robot-like a human is the best she or he will be at doing most jobs. So, it could be that the robots are most likely to hire the most robotic humans."
https://medium.com/@jannaq/the-robot-takeover-is-already-her...I find the whole gamified system to be bizarre and disheartening no matter which side of the table you're on.
To me, looking at modern tech interviewing is like comparing the gold standard OCEAN and the emergent HEXACO in personality surveys. Take the former on a bad day and it may leave the test taker feeling bad about themselves. The latter, much kinder and gentler in messaging around strengths and weaknesses.
That "by design" quality strikes me as missing from the entire tech interview system. If it weren't broken, this would not be a 7-year conversation updated yesterday:
If you slide from 50% to 99%, how do people feel about using ChatGPT? What is more honest: Many people here were hired when they were less than 100% qualified, and did very well in their new role. It has happened to me more than once.
I’ve gone public, been acquired by Google, and scaled solutions to tens of millions of users. I’m probably overqualified for your CRUD app.
Instead of tearing into their experience, my coworker is asking what you would use the X class for.
Drives me fucking nuts. Who memorized all the parts of a random, mostly unused .NET class.
I asked the coworker afterwards if he ever used said class, dude said no.
How is that a fair question if it isn’t even used here?
Luckily I have the contacts and experience to never have to.
Easy. They have nothing to lose because the jobs they are qualified for don't even pay enough to survive. You probably could have figured this out yourself.
Money, obviously.
Software jobs in particular are magic in this way - the pay is way above the average, and performance metrics are so poorly defined that one can coast for months doing nothing before anyone starts suspecting anything. Years, even, in a large company, if one's lucky. 80% of the trick is landing the first gig, 15% is lasting long enough to be able to use it as a foundation of your CV, and then 5% is to keep sailing on.
No, really. There's nothing surprising about unqualified people applying for software companies. If one's fine with freeloading, then I can't think of easier money.
(And to be fair, I'd say it's 10% of freeloaders, 10% of hard workers, and in between, there's a whole spectrum of varying skills and time and mental makeups, the lower half of that is kind of unqualified but not really dishonest.)
Center Stage is a feature where a device uses an ultra-wide camera, and then is supposed to track your _face_ as you move and shift around it it's field of view.
I find it most useful for FaceTime calls on Apple TV, where you can leave your phone near the TV, and it will automatically frame you sitting on the couch and will follow you as you shift around, etc.
There is a similar feature to what you're describing for FaceTime, but I don't think it has any cutesy name.
https://appleinsider.com/articles/20/09/25/how-to-use-faceti...
Had an interview take home assignment done by GPT and it was easy to spot after seeing dozens of solutions. Downside for the guy was - it didn’t work.
We have a small team of developers and you cannot hack metrics. You build and deliver what is in requirements or not.
If you don’t deliver we don’t even have to have a discussion because team reviews code, tests features and gives feedback quickly if someone is slacking.
The XKCD one can actually be easily hacked. Just spam the system and rate every comment as helpful. Classifier learns to accept everything. There's dozens of way to hack this one and undermine the actual goal. But it is a comic, and it is funny. Doesn't need to be realistic.
It's also why it's kinda annoying to do live interviewing trivia questions. Can I immediately answer what a partial template specialization is? Probably not, I never used them. Can I google it in 2 minutes and summarize it as as way for (often c++) template classes to bound some of the template arguments to values or pointers? Well, I just did. Should that cost me the interview? That's pretty much what I do on the job.
In the end, it's just a new way to "Google" the answer. After all, there isn't much difference between reading off an LLM response and just reading the Wikipedia page after a quick Google search, except for less advertisements.
I will say there are still some programming questions you can give that will stump the hell out of ChatGPT. In particular I took one online coding assessment where I used it and there was a question about plotting on a graph with code and calculating areas based on the points plotted that ChatGPT failed miserably at, but someone pretty good with math and geometry would find pretty tractable.
You're already asking Leetcode questions that are irrelevant for the job you're hiring for. What's the problem with asking one more to test for cheaters?
Instead, there would be tasks that can be completed using any tools available - Google, LLM, whatever. And candidates are rated on how well the task is done, and maybe asked a few questions to make sure they made decisions knowingly and not just copied the first answer off the internet.
This already exists and is called "take home programming assignment"
We’ve all got promotions by changing jobs in the last 6 months using this method.
You can be subtle about it if it’s already an area you kind of know.
People who are unwilling to say, “I don’t know, let me look into that,” are not fun to work with. After a while it’s hard to know what is fact vs fiction, so everything is assumed to be a fabrication.
At the end the assessor told me that I passed specifically because I said "I don't know". They purposely put questions on the test they didn't expect you to answer to see what you do when faced with an unanswerable question.
I've used that in my own life since -- I much prefer working with (and have a much more positive view of) people who are willing to say "I don't know".
If a candidate is trying to tap-dance or be vague around something to avoid admitting ignorance of it, that's a pretty large red flag.
And whether the interview is just asking definitions or silly certification questions, or things requiring deeper understanding.
During some interviews I’d give people access to a computer. If they could quickly find answers and solve problems, that is a skill in itself, but I could see what they were looking up. Sometimes that part would make or break the interview. Some people didn’t have a deep base of knowledge in the area we were hiring for, but they were really good at finding answers, following directions, and implementing them successfully. They would be easy to train on the specifics of the job. Other people couldn’t Google their way out of a paper bag, I was shocked at how bad some people were and looking up basic things. Others simply quit without even attempting to look things up.
> [If one assumes that the candidate] would have been able to perform the job duties I'm not sure why [they] should care.
This is what I mean; I can see why an interviewer thinks they've been cheated or that a candidate was dishonest but that doesn't mean that the interviewer even has a successful system for determining if a candidate can perform the job duties. A candidate who cheated -- from the perspective of the interviewer, I guess -- but still manages to adequately perform in their role very plainly did not cheat from a less biased perspective. What is that interviewer even thinking? How could that person have cheated?
That's not what anyone means when they say "cheating". Cheating means to violate the conditions and assumptions of an examination or contest.
For example, if a chess grandmaster uses an AI implant to win a game and gets caught, it doesn't make it OK if they could consistently win against the same opponent even without the AI.
I recall a Starcraft 2 match[0] involving a person with an apparently psychosomatic wrist injury that was only painful while they’re playing on stage. Their opponent was seeming to draw out a game they were losing in an attempt to trigger the pain; it was a viable strategy given the “best of” series they were playing. That’s certainly not going to be accounted for in the rules and one might believe that it’s an underhanded way to win. But both players are in the top echelons of game knowledge, experience, and skill; that’s the only reason either player made it to this particular match-up. The player with the wrist injury ultimately had it act up and lost the series.
Did the winner deserve to win? Should the other player be considered the better player? The assumptions of the game rules and what’s “fair” might be different per player; who’s right, who’s wrong, and why? What about when prize money is involved; that guy who won by the written rules just doesn’t deserve it because of unspoken rules? These questions don’t seem to have obvious answers, so of course I challenge assumptions.
0: I’m looking for the VOD I watched. Edit: I believe it was here: https://www.youtube.com/watch?v=DS2XIyNDlSA
There are nicer ways to express your meaning. I haven’t ignored anything.
These traits are often not offered by the employer. Why do I keep hearing people talk about the underhanded ways that companies try to obfuscate salary budgets if not because they’re dishonest? I certainly see that as dishonesty; where are they coming from to demand such honesty from their candidates?
They get honesty anyway but that doesn’t mean I can convince them of it. If a person wants to assume guilt in someone, that is often what happens. You may not have experienced a person power-tripping over you but that’s been a good portion of my life and it’s hard to miss the patterns in a modern job interview.
To be clear, I’m not advocating for one to be dishonest. The person using ChatGPT to supplement their knowledge is not being dishonest; that’s my claim. The interviewer feels like the candidate “cheated”. Oh well. Too bad the interviewer isn’t above pejoratives. Gotta call it “cheating” so they can dismiss the candidate as dishonest. How dishonest!
Taking it even a step further - your memory is imperfect, the degree to which you can accurately recall events is significantly poorer than most people believe, which leads to incidental lies. We call them mistakes, but from the outside perspective that's just a question of intent.
That being said, despite my pessimism towards human nature, I too value honesty. But, like everyone, I lie occasionally - and I note that you don't claim to not lie, nor to have never lied. I'd call it honesty on a best efforts basis.
This suggestion that a person who can adequately perform job duties could have even possibly cheated in their job interview is intellectually dishonest. If they had to cheat to get the job we should be looking at the interviewer. Why did the qualified candidate have to cheat? Why is whatever-they-did even considered cheating?
If they're qualified, they didn't have to cheat. If they're not, then they did. Either way, they're dishonest and that means they're not a desirable hire.
(Just rewriting to specify my understanding: If the candidate was qualified, they didn't have to cheat even if they did cheat. They could have simply not cheated and been selected by the merits of their qualifications.)
This argument relies on the false premise that an interviewer will always accurately determine a candidate's qualifications. That a candidate is not qualified to pass an interview is not the same that a candidate is not qualified for the job for which they're being interviewed.
But also, there are usually several-to-many applicants for a position that are all qualified, and by necessity most of them won't get the position.
Additionally, technical qualifications is only a part of what an employer is looking for. There are other things that are at least equally important -- how well the applicant would fit into the team, how trustworthy they are, etc. It's about a lot more than just technical skillset.
This is ultimately something I see as dishonest given the context of job applications. Employers generally expect a certain kind of perfection from job candidates, which they can’t manage to show of themselves. I understand that this isn’t an easy thing to solve -- nor even something that’s ever been solved -- but that should at least make it more understandable when an otherwise qualified candidate uses disallowed tools in their interview.
Perhaps the candidate’s real best option is to find a different company to work for but they may not be so privileged as to have a choice if their on-paper qualifications are lacking. Assuming their practicable qualifications are adequate, they may have good reason to bullshit through a bad interview. Additionally, finding a different company is pretty likely to be “same shit, different day”.
> But also, there are usually several-to-many applicants for a position that are all qualified, and by necessity most of them won't get the position.
Assuming they’ve qualified via an interview and there are particularly close candidates, pick the one who applied first. They’re admittedly qualified and further interviewing is just a means of discriminating in error-prone and possibly unlawful or immoral ways.
> Additionally, technical qualifications is only a part of what an employer is looking for. There are other things that are at least equally important -- how well the applicant would fit into the team, how trustworthy they are, etc. It's about a lot more than just technical skillset.
Fair enough. I would caution interviewers against judging too harshly or quickly. One can imagine many reasons an interviewee might choose or seem to lie during an interview while they are otherwise an honest person, ranging from stress to disillusionment to [cultural differences](https://news.ycombinator.com/item?id=39209794).
At the end of the day, filtering for liars and cheaters actually filters for bad liars and cheaters in addition to people who are a bit nervous or tired or stressed or cynical or just having a slightly off day; dishonest people who genuinely see nothing wrong with dishonesty get through just fine.
I don't think my friends disclosed that they were using it.
Jokes aside, something about LLM responses is very uncanny valley and obvious.
1) Select All (most likely followed by the copy) 2) Type the answer 3) Make an obvious mistake when they type else block, before the if
Thanks for the XKCD. I didn't realize how common this is. Now I'm even more annoyed that so many websites and reader apps force context menus or 'gestures' when you highlight, without a way to disable those context menus or gestures.
Ever go to school with a dyslexic that's using a ruler to expose one line of text at a time? Same thing....
(I also just select randomly sometimes. Not even quite sure why.)
I'm fidgety in general. If it isn't highlighting it's figure 8s with the mouse cursor.
i've actually been called out for it in a systems design interview, under the presumption i was copying my notes into another window, but was glad they called me out so that i could explain myself
Presumably you have tasks that you want performed in exchange for money? (Or want to improve your position in the company hierarchy by having more people under you or whatever).
I think that what will change is that doing interviews remotely will become rarer, in favor of in-person interviews.
Interviewing as a process sucks enough as it is. It should just be a culture fit filter that takes you all of 15 minutes to say yes or no to.
Technical interviews are lame and filter for people that are good at technical interviews, not people that are good at the job.
It truly does, and it sucks just as much for the employer as for the applicants. That's why I suspect that more interviews will be required to be in person: if it's too easy for someone to cheat, that makes everything suck even more for the employer and the employer is likely to adjust the process to minimize that suckage.
> Technical interviews are lame and filter for people that are good at technical interviews, not people that are good at the job.
Not automatically, but yes, bad technical interviews filter for people who are good at technical interviews. And too many interviews (technical or otherwise) are bad.
The number of experienced candidates I’ve interviewed just in the past few months who have trouble writing a for-loop in the language they’re “experienced” in might astound you.
Welders sometimes (always?) have to go to a certification center to demonstrate that they can actually perform the types of welds the job they’re applying for requires.
https://www.aws.org/Certification-and-Education/Professional...
In the current job market, however, lots of places are asking ridiculously hard verbatim leetcode questions in an attempt to filter out "bad candidates." Job seekers feel that too many places ask unfair questions (which is true) and employers feel that there are too many candidates that can't write genuinely simple programs (also true).
A candidate can do very well on personal and web project experience questions, and suddenly blank when you ask them how an http request is structured. Or what's CORS.
Then you dig further and discover a lot more thing about them that wouldn't have surfaced otherwise because hou assumed they knew all of that.
My best advice would be to never skip "dumb" and easy technical questions. You can do it very quick, and warn ahead that it's dumb questions but you ask them to everyone.
I see it as a different angle to get more information.
I'm not sure what specific questions you have in mind, but ChatGPT is almost certainly trained on a vast array of resumes and a diverse range of profiles, possibly even all of LinkedIn itself as well as other job boards. There is little to no reason why it wouldn't be able to make up an entire persona who is capable of passing most job interviews.
I always start interviews by asking them to explain their own projects. However, sometimes I'll find someone who's great at explaining projects they supposedly worked on in great detail, but then when given a simple coding problem they can't even write a for loop in their own top language.
That's a bit better than proxy interviews and people lip syncing, but not by much.
Sure, the point that superior tool use is a valid job skill makes some sense, but conceding your agency and higher reasoning to a machine which possesses none of these is to my mind not going to be beneficial to a business in the long run.
I think most people have been thinking that the interviews are mostly BS with little relationship to the job, which you simply have to get through.
Many, many people will cheat to the extent that they think they can get away with it.
It's a bit like many people cheat in school. (On classes they consider irrelevant, they might justify it that way. On classes relevant, they might justify it, that passing or their GPA is more relevant to their goals, than learning that material at that time.)
I think people generally don't believe a "you're doing a disservice to yourself" argument. They choose the tradeoff or the gamble.
Personally, I don't tolerate cheating, and I have a low tolerance for interview BS. Neither is the dominant strategy for the current field.
I suspect the way to deal with ChatGPT is to allow it. Expect the interviewee to use ChatGPT as a tool. Try out the interview questions beforehand with ChatGPT. Ask questions that ChatGPT won't be good and answering, like how a calculator is useless on a physics exam.
NOT to browse through looking for a solution from step 0.
In an open-book test, you have to know what you're looking for and roughly where to find it in the book. That implies some knowledge. With ChatGPT you could type the question verbatim and get a potentially right answer, without even understanding the answer at all. It is therefore unacceptable for use on any exam.
Assuming the interview is to determine somebody's programming chops, without the benefit of ChatGPT, you'll have to ask questions where ChatGPT is little to no help. This was the conclusion of the article.
Heck, at that point you aren't even measuring whether the candidate understood the question, nor their ability to communicate about it with prospective coworkers.
If there are any questions where "repeat whatever ChatGPT says" seems like a fair and reasonable answer, that probably means it's a bad question that should be removed instead. Just like how "I'd just check the API docs" indicates you shouldn't be asking trivia about the order of parameters in a standard library method or whatever.
That's a bit of a strawman: I didn't say anything about the ease/difficulty of the role being filled, and I implied rote memorization was not meaningful.
To reiterate, interviews should measure good data for choosing between candidates.
That's not happening when the given problem is solve-able by an LLM using a human as a proxy, everybody's just burning man-hours of company/applicant time on interview-theater that isn't useful for making a decision. (Well, not unless the hiring goals include "willingness to jump through hoops".)
If it is out in the open, with the chat/prompts available, you can ask other questions. You're not on your toes trying to catch a cheater. You're not assuming that the interviewee is lying or trying to scam you.
It may not be what the interviewer wants, because they would like to keep using their old interviewing strategies. However, it is what the business wants (or should want, assuming they want the most effective employees).
It will become a skill. In 1900 you'd interview a computer (a person who does math) by asking them to do math on paper. Now you'd let them write some code or use software to do it. If the applicant didn't know how to use a (digital) computer, you'd negatively rate them.
I don't love it, but we may reach the point where your skill at coaxing an LLM to do the right thing becomes a desirable skill and you'd negatively rank LLM-illiterate applicants.
Looking at LLM quality, we're not at that point for most fields.
Of course this will change in the future, with more interactive models, but people who use ChatGPT on the interviews make a disservice to themselves and to the interviewer.
Maybe in the future everybody is going to use LLMs to externalize their thinking. But then why do I interview you? Why would I recommend you as a candidate for a position?