I think it's likely ultimately a quest to find a way to make a mark, and he's gotten way out over his skis in the process.
As an aside, the conversation he posted seems much more advanced than anything I've seen publicly. Not sentient, but still quite impressive.
I don't think it is sentient without more evidence. But just wondering if not "thinking" when not questioned is a good reason it isn't?!?!
Still it's interesting! Obviously we have to program the system to stop after a sentence or two so the user can read it. What if we could program it to keep "talking to itself", eg speculating about possible continuations on both sides.
Talking to itself, speculating about continuations, etc - but not actually outputting them (else it's just a longer response). Instead, storing some parts in the buffer. Perhaps bifurcating. Even better, using some down time to summarise recent material and store the summary in the buffer. That's not a bad description of "thinking".
The whole motivating aspect of these models is "attention is all you need" to reduce dependencies from recurrence, which would otherwise halt this obscene scaling in its tracks.
We would put a small non-learning/non-neural interface on top of the system to implement these ideas. That interface could act like this:
* To ask for extra "thinking" text: after the underlying model stops typing, we output that text to the human user. But we then do the equivalent of pressing tab to request more text, and buffer that.
* To summarise some text (eg some of the thinking text from above), we can use another instance. We put it into summarisation mode, eg using the TLDR hack or any other method. Summarised text can be used as a prompt, or as output.
* We can bifurcate by copying instance state and starting a new instance.
These are pretty basic ideas, probably already in use, but I think they show how we can expand the system from a kind of instantaneous stimulus-response to something more interesting.
Hopefully it's clear this is not equivalent to sleep(10). In my view it doesn't make the system more intelligent, rather it allows the system to use its existing abilities more fully.
(edit: another aspect we could control would be switching the system between high-temperature modes and low-temperature modes, in different instances, and depending on what we're trying to achieve. This relates a bit to the "speculation" comment made above by another user.)
I don't think it should be reasonable to compare it with human reasoning where we think constantly and our brain operates even while we sleep: it is not a human being after all
We're all suspectible to this. The more time you spend with something, and the more time you spend wanting to see things, the more likely you are to catch a pattern that isn't really there. Have we achieved anything close to AGI? No, of course not. Are more and more people going to be fooled by what we have achieved? Yes, absolutely. The so-called Turing test is actually shockingly easy to pass.
I don't think Lemoine s an idiot. Maybe he's mentally ill; maybe he's not, but has been taken in for other reasons, just as people without biological mental illness are still susceptible to being taken in by cults.