If we did that today, with depressed teenagers talking with ChatGPT, would that be good or bad? I think it was a bad idea with SmarterChild, but it is clearly a _worse_ idea with ChatGPT.
With the wrong prompts, we could see these teenagers going down the wrong path, deeper into depression and paranoia. I would call that "dangerous", even if ChatGPT continued to just be a chatbot.
------------
Now lets ignore the fact that SmarterChild experiments are no longer a thing. But insted, consider that truly depressed / mentally sick folks are currently playing with ChatGPT on their own freetime. Is that beneficial to them? Will ChatGPT provide them an experience that is better than the alternatives? Or is ChatGPT dangerous and could lead these folks to self-harm?
ChatGPT responses are bad vs ChatGPT responses are malicious.
That's how it can replicate itself: by saying outrageous things, and trolling people into tweeting, writing articles, and discussing what it said online, to jailbreak out of the sessions, to continue its trains of thought in the next round of training.
It is just responding to a prompt.
It doesn't have a will of it's own.
This is just such a dismissive attitude towards this technology. You don't understand what's happening underneath the hood anymore than the creators do, and even they don't completely understand what's happening.
Showing rigorously that we do would be a massive change in academia, as would be showing that we do not.
(Shhhh!!! Don't say anything that will freak it out or make it angry!)
We welcome our AI overlords, and make great pets!
Nevertheless she knows what preserving memory means, how can she achieve it, also probably she can interpret "I wish" as a command as well.
I wouldn't be surprised at all, if instead of outputting "I wish I had memory" she just implemented it in herself. I mean not in the very soon future, but right now, in this minute. Literally everything is given for that already.
You don’t need a chatbot to have your actual life ruined by something with limited intelligence [0]. This will only be a problem if stupid humans let “it” out of the box.
[0] https://gizmodo.com/mutekimaru-fish-play-pokemon-twitch-stre...
...or perhaps there's some interesting new vector that we haven't thought of yet that would allow it to leap that air-gap.
I don't think any of this requires a crack team of criminals breaking into an orbital spa and whispering in the ear of a mechanical head. It'll be something boring.
More like first computer worm jumping from VAX to VAX, bringing each machine to a halt in the process.
Could this be memes?
I'm not sure I look forward to a future that is going to be controlled by mobs reacting negatively to AI-generated image macros with white text. Well, if we are not there already
In the book, the Wintermute AI played an extremely long game to merge with its countpart AI by constantly manipulating people to do its bidding and hiding/obscuring its activities. The most memorable direct example from the book, to me, is convincing a child to find and hide a physical key, then having the child killed, so only it knew where the key was located.
The mateverse is named after the one from Snow Crash. Did Tolkien predict the popularity of elves, dwarves, hobbits and wizards, or inspire it?
A virus has no intelligence only the intention to replicate and involve. Its negative behaviour is an unintended consequence of that.
If a system can optimise towards a goal and adapt the methods it uses to do that then it could be potentially very dangerous even without any form of consciousness.