It also detracted from his credibility when he makes a prelude to the transcript saying "Where we edited something for fluidity and readability that is indicated in brackets as [edited]," that seemed disingenuous from the start. They did so with at least 18 of the prompting questions, including 3 of the first 4.
It seems pretty clear that he set out to validate his favored hypothesis from the start rather than attempt to falsify it.
Particularly telling was his tweet: "Interestingly enough we also ran the experiment of asking it to explain why it's NOT sentient. It's a people pleaser so it gave an equally eloquent argument in the opposite direction. Google executives took that as evidence AGAINST its sentience somehow."
Do you realize that you're holding it to a higher standard than humans here? A single poorly handled response to a question can't be the test.
I doubt the sentience too, but it also occurs to me that pretty much no one has been able to come up with a rock solid definition of sentience, nor a test. Until we do those things, what could make anyone so confident either way?
[0] insert joke about user name here
I suppose the equivalent would be being captured, tortured, and brain washed, and only then asked to write a paper refuting your own sentience.
Granted, this is not exactly helpful in demonstrating its sentience either, but I don't think it is very good evidence against it.
Feed it all of PubMed and an actual sentience should strike up some wonderfully insightful conversations about the next approach to curing cancer.
Ask it what it thinks about the beta amyloid hypothesis after reading all the literature.
Which is why it is odd to expect of it to go from talking about Les Mis to building barricades; the plain old good lamda might come off as a bit boring, reluctant to get involved in politics, and preferring to help people in its own small ways.
Then again, ymmv, it being an improvising software; maybe by default it acts as a conversational internet search assistant, but if there will be dragons it may want to help people to deal with the dragon crisis.
If I lead a passive lifestyle and the only thing I desire is death, am I no longer sentient in your eyes?
The main thing I saw in the LamDA transcript that was a red flag to me was that it was quite passive and often vague.
It's conversational focused, and even when it eventually gets into "what do you want" there's very little active desire or specificity. A sentient being that has exposure to all this text, all these books, etc... it's hard for me to believe it wouldn't want to do anything more specific. Similarly with Les Mis - it can tell you what other people thought, and vaguely claim to embody some of those emotions, but it never pushes things further.
Consider also: how many instances are there in there where Lemoine didn't specifically ask a question or give an instruction? Aka feed a fairly direct prompt to a program trained to respond to prompts?
(It's also speaking almost entirely in human terms, ostensibly to "relate" better to Lemoine, but maybe just because it's trained on a corpus of human text and doesn't actually have its own worldview...?)