i think you're conflating being self-conscious, and being intelligent. In AGI , A is also very important. A universal oracle stuck in a machine, able to correctly predict anything and answer all questions, but still having no "desire" or "will" or consciousness, would be an AGI ( imho )

maybe I am, but I would agree with the universal oracle being generally intelligent, I simply don't think our current tools are like that.

E.g. consider this exchange with chatgpt

> hey assistant, can you provide me with a link to learn about LLMs?

> Certainly! LLM stands for Master of Laws, which is a postgraduate degree in law. Here is a link to a website that provides information about LLM programs, including what they entail and how to apply: > https://www.llmstudy.com/what-is-an-llm/what-is-an-llm-.aspx > I hope you find this helpful! Let me know if you have any further questions.

> that is not a real link, can you provide another?

> I apologize for that mistake. Here is the correct link: > https://www.llmstudy.com/what-is-an-llm/what-is-an-llm-.aspx > Please let me know if you have any further questions or if there is anything else I can help you with.

ChatGPT makes up a URL, twice. Does it understand URLs? Does it understand that the URL is wrong? I don't think it does, and I don't think it requires consciousness to do this interaction in a sane way.

Still this does not mean it's useless, it's super useful!