The existing ones can write code. Suppose we get ones that can write better code and engage in self-improvement. Now you have something which is a billion lines of code and is under constant self-modification. We no longer have any idea how it works, but it can feign consciousness in much the same way as existing LLMs. What's the test for if it actually achieves consciousness?

I can give you one that a human being in good health will pass and a rock will fail. Can you give me one that an AI feigning consciousness will fail?

Or if you use a less selective filter then it goes the other way. Exclude philosophy texts but not Facebook posts and it would still have what it needs to emulate emotional text regardless of whether it has ever seen a formal discussion of it. And it could plausibly extrapolate from that to an analytical discussion, having seen both that and analyses of other subjects, without ever feeling anything itself.