Not AI’s pets, but oligarch’s pets

No, the AI will work for us, it's our pet.

Why would something smarter than us be our pet?

Because we will have full control over it. It will do what we ask. The AI has no motivation or personality, it's just a tool used by humans.

Maybe it's time to start differentiating between artificial intelligence and artificial consciousness? Surely I can't be the first one to think of this...

I agree with the sentiment, but I also don't think such differentiation is possible, because we have no testable definition of consciousness. I genuinely look forward to the day we discover/understand the physical processes underlying our own consciousness, and how to detect/disprove it in machines, but until then... it's unfortunately unfalsifiable.

Sure, but for now, we can have a crude differentiation between an AI - a tool which has to be given an objective to work towards, and an "AC" - an entity (an intentionally vague noun) which can devise its own objectives and motivations and work toward them.

Agentic AIs are more useful than mere tools, therefore they will be built, and agents have motivation. The alignment problem is how to give them exactly the right motivation, such that they keep us as pets instead of killing or factory farming us. Like humans keep cats.

And what if a human tells it to control other people? What if a rogue third party terrorist group replicates OpenAI's latest and greatest in the year 2030, but explicitly tasks it with hurting people instead of helping them?

I don't agree with "we will have full control over it." I'm more concerned about how much control it will have over us.

That's completely backwards. Pets don't have to work. Try to convince my cat to do something useful. I'd switch places with him in heartbeat if could. If an AI, or anybody else for that matter, offers to make me their pet without me having to grow fur and losing the opposite thumb (but I still could learn to purr if needed), I'd be even more for it...