Re:Testing how hard it is to cheat with ChatGPT in interviews
by fragmede on Wednesday January 31, 2024 @07:36PM
(#39211532)
There are ChatGPT resistant questions you can ask. ChatGPT recognizes the question but doesn't actually think about it, so if you give it the river crossing problem (farmer, fox, sheep, and grain need to cross a river) but tell it the boat can take all the items, it won't actually read those details and blithely solve the problem the expected way. Give candidates a problem that's trivially solvable if you actually read the question and see if they try and solve it the ChatGPT way.
[ Parent
| on HN ]
Re:Testing how hard it is to cheat with ChatGPT in interviews
by hirsin on Thursday February 01, 2024 @12:04AM
(#39212872)
Indeed - with 3.5 at least it does fail the easy mode riddle
https://chat.openai.com/share/b3761807-551d-4cfc-b291-6d37ee...
Re:Testing how hard it is to cheat with ChatGPT in interviews
by fragmede on Thursday February 01, 2024 @03:19AM
(#39213798)
it's a fun problem to explore, and gpt-4 doesn't do any better. swapping in other things doesn't help because it internally recognizes it as the river crossing problem and proceeds to solve it normally. I was able to get it to two shot it with a lot of coaching but yeah, it's a trip.
Re:Testing how hard it is to cheat with ChatGPT in interviews
by ric2b on Thursday February 01, 2024 @10:57AM
(#39217269)
The downside is that you're now wasting interview time on the river crossing problem instead of actually relevant questions for the job you're hiring for.
Re:Testing how hard it is to cheat with ChatGPT in interviews
by fragmede on Thursday February 01, 2024 @12:19PM
(#39218423)
Don't literally use the river crossing problem, but it's existence implies there is a form of question that ChatGPT will solve for where someone actually reading the prompt can solve trivially but someone using ChatGPT will get stuck on.
You're already asking Leetcode questions that are irrelevant for the job you're hiring for. What's the problem with asking one more to test for cheaters?