> And learning means it will acquire new motivations

This conclusion doesn't logically follow.

> Fixed motivation thing is simply a program, not AI

I don't agree with this definition. AI used to be just "could it solve the turing test". Anyway, something with non-fixed motivations is simply just not that useful for humans so why would we even create it?

This is the problem with talking about AI, a lot of people have different definitions of what AI is. I don't think AI requires non-fixed motivations. LLMs are definitely a form of AI and they do not have any motivations for example.

Disclaimer - I don't consider current LLMs as (I)ntelligent in the AI, so when I wrote AI in the comment above it was equivalent to the AGI/ASI as currently advertised by LLM corpos.

Consciousness, intelligence, and all these other properties can be and are mutually exclusive. What will be most useful for humans is a general intelligence that has no motivation for survival and no emotions and only cares about the goals of the human that is in control of it. I have not seen a convincing argument that a useful general intelligence must have goals that evolve beyond what the human gives it and must be conscious. What I have seen are assertions without evidence, "AI must be this way" but I'm not convinced.

I can conceive of an LLM enhanced using other ML techniques that is capable of logical and spatial reasoning that is not conscious and I don't see why this would be impossible.