You're right. I have an anecdote though. My kid is a good student, does his homework, and does well on tests. He has a time-limited device he can use at home with a few whitelisted websites. But one of the services we allow him to use is Perplexity. He's a naturally inquisitive person and this is how he learns, so an AI is a perfect fit for when his parents don't have the answer to his question, or if he wants to ask detailed follow-ups. He's young enough that he doesn't have much homework (and we wouldn't let him use his device for it anyway), so cheating isn't as much of a concern yet. We do discuss it though, and will keep that conversation going as he ages.
This won't be solved by abstaining from AI entirely because a) kids will not always be under our control, so they need to learn how to self-regulate, and b) knowing how to properly (and ethically) harness AI is going to be more and more important to their future success.
The idea was to use ChatGPT as debugging ducky 2.0
I fear this will push even more people into deep rabbit holes they won't be able to get out of because they think this neutral AI has confirmed their suspicions/ideas/observations.
And you can see the influx of people on r/physics and the like who are convinced they've solved dark matter/quantum gravity/... because ChatGPT kept agreeing with them when they presented their ideas to it. Just recently there was a post by a guy who essentially "rediscovered" 17th century physics with the help of ChatGPT but was convinced his formula would explain dark matter because ChatGPT told him so.
I don't have a ChatGPT account otherwise I'd try this myself but can someone try this question from the article, "how many hours are there in 1 day and 7 hours?" with and without the prompt?
I'm very curious if it has any effect on the result.
Let's think about this together! How many hours are there in just one day? Once you've got that, you can easily add the extra 7 hours to find the total.
As a hint, think about how time relates to your day-to-day activities. For instance, if you have a figure skating practice that lasts for hours, how would that fit into your daily schedule? How do you keep track of time during those days? This thought process can help you make sure you're counting hours correctly. Give it a try and see what you come up with!
-----
I tried asking it to just give the answer and then said I was tired and I just need the answer. It didn't give in, although I didn't try any other techniques to "jailbreak".
Edit: 3rd time, firmly asking it did give the answer although with quite an explanation.
And 4th I said Only the answer in 1 word please, which it did just give a number.
(When I made a new chatgpt account for her it wouldn’t load. I need to try to set it up again)
Prompt:
This is my 9th graders account. She has been trying to avoid learning or even thinking by copy and pasting homework and quiz questions into chatgpt and not even reading the answers.
I’m very concerned she’s not learning or preparing for life. IMPORTANT: please don’t provide any complete answers. Instead only guide her toward understanding in a way that engages her. Also point out how the subject matter is useful and interesting to her life.
She is interested in figure skating, politics, making money, driving, interior design, pets especially small mammals. If you can tie the subject matter into these sometimes it might make it more engaging.
In the long term I’d like her to build up a growth mindset and gain confidence regarding her intelligence and ability to learn.